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ABSTRACT 
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This thesis assesses the capability of the Logistics 
Management Decision Support System (LMDSS) to meet the 
information needs of Naval Air Systems Command (NAVAIR) 
logistics managers based on surveys of logistics managers 
and interviews with LMDSS program representatives. 

The LMDSS is being introduced as a tool to facilitate 
action by NAVAIR logistics managers to reduce the life cycle 
support costs of aviation systems while protecting 
readiness. We conclude the LMDSS does not meet the 
definition of a Decision Support System due to the lack of 
modeling capabilities. The LMDSS architecture and 
capabilities meet the information needs of surveyed 
logistics managers and support Affordable Readiness 
initiatives which are the means by which NAVAIR intends to 
reduce life cycle costs while sustaining aviation system 
readiness levels. Lack of modeling, graphics and 
sensitivity analysis capabilities limits identification, 
analysis and comparison of Affordable Readiness initiatives. 

We recommend modeling tools and graphics capabilities 
be incorporated as part of the LMDSS application. We 
further recommend that initiatives to improve data validity 
be expedited and that Maintenance Level 3 detail cost data 
be provided. Recommendations are made for further research. 
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I. 



INTRODUCTION 



The Logistics Management Decision Support System 
(LMDSS) is being introduced to help Naval Air Systems 
Command (NAVAIR) logistics management teams reduce the life 
cycle support costs of aviation systems while protecting 
readiness. Recently LMDSS identified a "bad actor" in the 
F-14 aircraft avionics system. It isolated the Central 
Signal Data Converter (CSDC) and identified a replacement 
with similar functionality at a lower cost over the life of 
the avionics computer. A former F-14 APML stated without 
hesitation that the $42 million cost avoidance could not 
have been found without the LMDSS prototype system in place. 
(NAVAIR 7.0, 1997) Was this an isolated incident? Can we 
expect outcomes like this in the future from the LMDSS? 

Operating and Support (O&S) costs, that account for 50 
to 60 % of the Navy' s Total Obligation Authority, are 
escalating as aircraft age and compete for the same limited 
resources. (NAVAIR TEAM, 1998) The Navy must improve 
business processes to reduce costs in order to secure the 
resources needed for investments in modernization and 
recapitalization. Cost-reduction initiatives are driving 
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program managers to treat O&S costs equal to other 
performance criteria. 

Reducing life cycle support costs depends upon 
effective logistics management and planning that, in turn, 
depends upon tools to support decision-making. The LMDSS is 
a tool to reduce life cycle support costs of aviation 
systems. It is a Naval Aviation Logistics Data and Analysis 
(NALDA) Phase II application that incorporates data from 
existing maintenance, flight, cost and material databases 
into a structured decision making process. 
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II. NAVAL AVIATION PROGRAM MANAGEMENT 
AND LOGISTICS DECISION PROCESSES 



A. RESPONDING TO THE POST COLD WAR PERIOD 

The LMDSS is a tool designed to support the Program 
Managers, Air (PMAs) reduce life cycle support costs by 
supporting decision-making. The Navy must find ways to 
improve business processes to reduce costs in order to 
secure resources necessary to make investments in 
modernization and recapitalization required to carry out 
future missions. 1 NAVAIR is responding to the post cold war 
period in several ways (NAVAIR 3. 6. 1.1, 1998). First, by 
eliminating military-unique requirements and procedures that 
drive up acquisition costs_ the cost of aircraft systems and 
associated support is reduced. Second, new policies are 
removing the impediments to getting state-of-the-art 
technology into Navy aircraft and related systems. Advanced 
technology has proven a true force multiplier in the 
development and deployment of weapons systems (Hickock, 

1997; Fox, 1997). Third, firms that traditionally produced 

1 See www.acq-ref.navy.mil/pdf/abc.pdf, Navy Acquisition 
Reform for additional readings on modernization and 
recapitalization efforts. 
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goods primarily, if not solely, for the Department of 
Defense are encouraged to diversify into commercial markets. 
Fourth, new policies and strategies are targeting the 
reduction of operating and support (O&S) costs. The large 
consumption of resources in this area threatens Navy 
modernization and recapitalization efforts (Hickock, 1997; 
Fox, 1997). 

O&S costs account for between 50 and 60 % of the Navy's 
Total Obligation Authority (NAVAIR TEAM, 1998) . These costs 
are escalating as aircraft age, competing with investment 
requirements for the same limited resources and thereby 
hindering improvements to infrastructure. Cost-reduction 
initiatives are changing the focus of program managers who 
must now treat O&S costs as they do any other performance 
criteria; systems must be affordable and supportable as well 
as meet operational requirements. 

B. AFFORDABLE READINESS 

Affordable Readiness is the means by which NAVAIR 
intends to significantly reduce O&S costs while sustaining 
requisite readiness levels. The resulting cost savings and 
cost avoidances can then be directed toward modernization 
and recapitalization. The objective is to meet required 
mission performance and ensure safe operations at the lowest 
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ownership cost. Previously the focus of program managers 
was centered on schedule and the projected average unit 
procurement cost with secondary interest on projected 
operations and support objectives. The shift from Design to 
Cost to Cost As an Independent Variable is a philosophical 
shift. The previous approach resulted in maximized 
performance at nearly any cost. 2 In general, ownership 
costs can be measured in terms of manpower, materials and 
resources. Readiness, availability, operating time, turn- 
around-time, and other similar metrics measure performance. 
Proposed Affordable Readiness Metrics are listed in Figure 
1 . 

Affordable Readiness is a business practice with four 
inter-related elements: flexible sustainment, sustained 
maintenance planning, rightsourcing, and total cost of 
ownership. Appendix A describes each element. Analysis of 
naval aviation O&S costs reveals six major drivers that the 
program management team can influence by implementing 
Affordable Readiness: maintenance concept, inventory, 
manpower, technical data, infrastructure, and warranties 
(NAVAIR 3. 6. 1.1, 1998). Continuous review of in-service 
weapons systems to adjust the maintenance structure based on 

2 See Land, 1997; Kausal, 1996 for a historic perspective on 
CAIV . 
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Figure 1 Affordable Readiness Proposed Metrics. After NAVAIR 3.6, Affordable Readiness 
Brief, June 1997 and Affordable Readiness Web Site 



fleet feedback concurrently optimizes operational 
requirements and provides opportunities to eliminate 
unnecessary costly activities. Reliability Centered 
Maintenance (RCM ) 3 analysis and Logistics Engineering Change 
Proposals (LECPs) processes help to achieve better inherent 
reliability, target technology insertions, and avoid 
obsolescence. Better inventory management and repair 
process analysis can reduce out of service time for 
aircraft, spares, and support equipment. Smart decisions 
early in the acquisition planning process can reduce 
material and manpower requirements to support an aircraft 
system throughout its total life cycle. Partnerships with 
industry, consolidating capabilities, the use of digitized 
data, single process initiatives, reinvention initiatives, 
reliability warranties, and integrated diagnostics are some 
of the many cost-eff ective • initiatives . Program managers 
must make intelligent trade-offs between performance and 
life cycle costs. The decision support systems must support 
the PMA developing and analyzing alternatives. 



3 See www.nalda.navy.mil, NAVAIR Logistics, Affordable 
Readiness Link. 
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C. PROGRAM MANAGER ROLE IN AFFORDABLE READINESS 

Figure 2 depicts organization responsibilities as they 
pertain to Affordable Readiness. Affordable Readiness is a 
management approach being implemented within the existing 
organization structure. 4 The PMAs are responsible for 
developing plans to implement and execute Affordable 
Readiness; identifying specific reduction targets and 
metrics for tracking progress; setting priorities and making 
investment trade-offs; directing the actions of supporting 
teams like Fleet Support Teams and Integrated Product Teams; 
interfacing with support environments such as policy, 
process, facility and infrastructure organizations to 
develop optimal policies and processes; reporting actions 
taken and results; and obtaining fleet feedback on how the 
system is performing. Assistant Program Managers, Logistics 
(APMLs) advise PMAs on logistics matters. The program 
management office is where the rubber meets the road in life 
cycle management and total cost of ownership analyses. It 
is here that the manager who has both authority and 
responsibility is held accountable for effective and 
efficient allocation of resources. The PMA must balance 

4 See www.navair.navy.mil, NAVAIR for additional readings on 
the NAVAIR TEAM history and organization. 
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Figure 2 Affordable Readiness Organization Responsibilities. After SECNAVINST 5400. 15A 
and NAVAIR 3.6 Affordable Readiness Brief, June 1997 



short and long-term objectives and vie for critically 
limited resources. Additionally, he must do so while 
working within a labyrinth tangled with political and 
bureaucratic processes and pressures. 

D. INTEGRATED PRODUCT AND PROCESS DEVELOPMENT 

PMAs manage within the context of Integrated Product 
and Process Development (IPPD) which is a management process 
that integrates all activities from product concept through 
production/f ield support. It uses a multi-functional team 
to optimize the product and its manufacturing and 
sustainment simultaneously to meet cost and performance 
objectives. IPPD evolved from concurrent engineering and 
the philosophies of quality management. It is a system 
engineering process integrated with sound business practices 
and common sense decision making. 

Integrated Product Teams (IPTs) are the means through 
which IPPD is implemented. Appendix B describes the three 
types of IPTs. These cross-functional teams are formed to 
deliver a product with common performance objectives using a 
common approach for which they hold themselves mutually 
accountable. Members of an IPT represent the technical, 
manufacturing, business, and support organizations that are 
critical to developing, procuring, and supporting the 
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product. Team members work together to achieve the team's 
objectives . 

E. LOGISTICIANS AND INTEGRATED PRODUCT TEAMS 

As functional area experts logisticians have special 
responsibilities because they bring special knowledge and a 
special point of view to the effort. The degree to which 
these experts are willing to share their knowledge and point 
of view will determine their value to the team. In addition 
to providing an expert opinion, experts play an important 
training role on the team. By sharing their expertise, they 
educate fellow team members to the not-so-obvious 
implications of programmatic decisions and actions. Team 
member involvement includes active participation, effective 
communication, challenging requirements that do not make 
sense, and paying attention to detail. For the logistician, 
dedication to these principles can make the difference 
between fielding a costly, ineffective, inefficient system 
or an optimal one. Optimal solutions are often. a result of 
well-researched opinions, constructive conflict resolution, 
and tenacity. 

Because with few exceptions most of the cost of a 
program is in the cost of ownership, i.e. the support of the 
system throughout its operational life, the logistician can 
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make major contributions to the acquisition of a cost- 
effective system. While dealing with short-term problems, 
the logistician must also think about problems that will 
arise in the future. For example, increased environmental 
awareness and legislation has increased the difficulty and 
cost of demilitarization and disposal of systems. An 
unreliable system with poor maintenance support will drive 
the need to procure costly spare components that may or may 
not be available. Identification of such problems early in 
the concept exploration phase of a program can help avoid 
serious consequences later in the program's life cycle. 

The logistician' s role on an IPT depends on the type of IPT 
it is. On a higher level IPT not directly focused on 
support issues the logistician should be concerned with 
identifying and highlighting the long-term logistics 
implications of various program issues. He may then form a 
supportability IPT to focus on mitigating the effects of 
those issues on the supportability of the system. At the 
program level the logistician should be more concerned with 
influencing the design of the system and the design of the 
support structure. An important responsibility of the 
logistician on an IPT is to help the team create 
supportability performance requirements that are 
quantifiable and testable so that the decision-makers can 
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gain insight into the operational suitability of the product 
and the logistic planners can plan for the support of the 
item. 

The Fleet Support Team (FST) is an example of an IPT 
directed to interface with the fleet, identify problems, and 
develop solutions. The focus of these teams is on all 
aspects of life cycle support for a system from when it is 
fielded until it is disposed. Within the context of 
Affordable Readiness, the FST is the center for identifying 
initiatives, performing analyses, and developing action 
plans. The resulting action plans can include investments 
(Engineering Change Proposals) , process improvements 
(consolidated maintenance activities) , or innovative support 
solutions (commercial or joint service support equipment) . 
IPTs, including FSTs, are not standardized. Each PMA 
determines how he will manage his program. Some teams are 
highly specialized while others cross diverse functional 
barriers. Some FSTs are organized within the program office 
while others are organized within the aircraft controlling 
custodian or depot maintenance engineering support 
organizations. Accordingly, the APML has different 
relationships with teams, both within and beyond the program 
office, as determined by individual program office 
organization. Additionally, program offices use a number of 
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different sources for logistics support. Some program 
managers rely heavily on Navy personnel assigned within the 
program office, others rely on government employees from a 
logistics competency group in NAVAIR , 5 while some contract 
out to commercial sources for logistics analyses and 
recommendations . 

F. CHANGE FROM CHECKLISTS TO INTEGRATED 
FLEXIBILITY 

Because acquisition program management is tailored to 
meet individual program needs, the challenge of supporting 
information systems is to gather and present data in an 
equally flexible manner. The data must reflect performance 
and supportability metrics in a meaningful, effective, and 
efficient way. 



5 Within NAVAIR' s matrix organization in which team members 
report to both a functional leader and a program leader, 
competency leaders are responsible for providing skilled, 
knowledgeable members for IPTs and for managing the 
processes by which these personnel support the teams. The 
Logistics Competency Center's mission is to provide the 
people, skills, knowledge, processes, facilities, and 
equipment required to manage and perform the planning, 
development, acquisition, integration, and delivery of all 
integrated logistic support elements necessary to affordably 
design, support and maintain weapons systems throughout the 
program's life-cycle. Supporting missions include technical 
publication development, logistics elements integration, 
affordability studies, and engineering technical services to 
name but a few. 
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Traditional Integrated Logistics Support (ILS) used a 
step-by-step analytical process that defined all the 
logistics and maintenance tasks, resources, and requirements 
necessary to establish and sustain an effective support 
program over the life cycle of a program. The logistics 
community depended on ILS products generated from the 
application of the Logistics Support Analysis (LSA) process 
as stipulated in MIL-STD-1388-1A. Now as DoD 5000. 2-R 
stipulates, supportability factors are to be integrated 
elements of program performance specifications, but support 
requirements are not to be stated as distinct logistics 
elements. Instead they are to be stated as performance 
requirements that relate to a system's operational 
effectiveness, supportability, and life cycle cost 
reduction. The challenge to the PMA is to develop a 
performance-based statement of work that includes 
supportability metrics in addition to the usual 
operationally oriented performance goals. Programs must be 
able to be evaluated on specific performance metrics 
describing the relation of cost-of-ownership with other 
parameters such as mission performance, safety and 
availability/readiness. Appendix C describes four factors 
that must be considered in establishing supportability 
requirements . 
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MIL-HDBK-502 offers guidance on acquisition logistics 
as an integral part of the systems engineering process. The 
information is applicable, in part or in whole, to all types 
of material and automated information systems and all 
acquisition strategies but it is not a "cookbook" approach 
to acquisition logistics. It is intended to accommodate the 
vast, widely varying, array of potential material 
acquisitions and provides general guidance to logisticians 
on how to perform certain aspects of their jobs. 

NAVAIR has a companion "Contracting for Supportability 
Guide." This guide identifies five steps (Appendix D) to be 
used to establish a supportability strategy for acquisition 
programs for new systems, major and minor modifications or 
upgrades, and commercial and non-developmental items. 
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III. LOGISTICS MANAGEMENT DECISION 

SUPPORT SYSTEM 



A. HISTORY AND ORIGINAL REQUIREMENTS 

The LMDSS requirement evolved from NAVAIR and Aviation 
Supply Office (ASO) initiatives to assess the logistics 
health of programs using standard metrics of readiness, 
supportability, and cost. In 1991, the effort was expanded 
into a requirement to develop a decision support system 
which would be the primary tool for APML/Weapon System 
Managers (WSMs) to achieve a "continuous, measurable 
reduction in life-cycle costs while maintaining operational 
readiness and sustainability." (LMDSS Req. Doc., 1993) 

The driving requirement behind LMDSS was the need for a 
tool to facilitate measurement to plan and identification of 
targets for cost reduction. This would be accomplished 
through use of a software package operating on several key 
databases organized in a relational environment. The key to 
successful operation of the DSS rested with the integration 
of diverse databases into a central repository. This 
integration would result in an immediate, precise response 
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to queries, regardless of the type data requested or its 
origin . 

Access to logistics data in a relational environment 
allowed a level of analysis that was impossible in personal 
computer based systems. Where analytical requirements 
involved databases outside the repository, access would be 
made automatically as a standard function of LMDSS. The 
repository would encompass data from all three maintenance 
levels, the supply community and selected sources of 
specialized information. To speed access, the system was to 
be established using both local and wide area 
interconnection techniques. The system was to be designed 
to accommodate managers and analysts who were not 
necessarily Automated Data Processing (ADP) experts. 

The NAVAIR development team determined that to provide 
maximum utility, LMDSS needed to provide both a structured, 
modular approach and an unlimited ad hoc query capability. 

A requirement for an extensive repertoire of Statistical 
Process Control and traditional charting available on a 
semi-automated basis was established to compliment both of 
these capabilities. 

The plan was for the data repository to be located at 
ASO to provide all Naval Aviation maintenance personnel, 
engineering personnel, supply personnel and procurement 
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personnel a "common and integrated sheet of music." (LMDSS 
Req. Doc., 1993). Provisions would be made to integrate 
Streamline Alternative Logistics Transmission System (SALTS) 
data with traditional Aviation Maintenance and Material 
Management (AV3M) reporting when available. The objective 
was to incorporate all necessary AV3M and Uniform Inventory 
Control Point (UICP) data, so that it could be manipulated 
directly by the logistician, to support detailed causative 
research. The NAVAIR development team also desired summary 
data, where refined and available. However, the focus was 
to remain on ability to support detailed research in a 
relational database management system environment. 

The LMDSS software was to be divided logically into 
five modules of 1) Candidate Selection; 2) Problem 
Isolation; 3) Ad hoc Queries and Special Summaries; 4) 
Evaluation and Selection of Alternatives and 5) 
Implementation and Status Tracking. 

The original requirements for LMDSS assumed a IBM RISC 
System/6000 Model 970 machine located at ASO hosting a 
massive - hundreds of gigabytes - database and regional IBM 
RISC System/6000 machines located at each Naval Aviation 
Depot (NADEP) and selected Naval Air Warfare Center (NAWC) 
activities. The regional machines would not house the 
extensive data held in the NALDA ASO IBM RISC System/6000 
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machine. The regional machines would tap the ASO database 
resource via remote procedure calls. Connectivity between 
the RISC machines would be provided by either direct 
connection of each RISC system to the NAVNET Internet or 
directly to the DDN MILNET. It would employ a client/server 
distributed architecture linking these computers via TCP/IP 
in a WAN and LAN environment. ASO LMDSS customers would 
directly interact and conduct X Windows client/server 
sessions with the ASO RISC machine via LAN and TCP/IP. All 
other LMDSS customers would directly interact and conduct X 
Windows client/server sessions with their respective 
regional RISC system via LAN and TCP/IP. It was planned as 
an information system requiring direct LAN connectivity. 

The original system was not planned to support PCs or 
workstations in stand-alone mode, nor support connectivity 
via modem. 

All programming was to be done in Ada. The operating 
system for the RISC machines was to be IBM AIX and Oracle 
was to be the database. Rapid prototyping was used for the 
software development methodology. The prototype system is 
running on a RISC machine located at ASO. 

By 1995, it was determined that Ada was unsuitable as a 
host language. HTML and PERL were used to request and 
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display reports. X Windows and C were being used for 
programming complex displays. 

In 1996, the scope of the LMDSS underwent substantial 
change. The LMDSS database was expanded to essentially 
become NALDA Phase II (NAVAIR 3.6.2, 1998). The LMDSS did 
not start out as the heart of NALDA II but at that point, 
that is what the system became. Appendix E provides a 
detailed discussion of the evolution of NALDA and the 
composition of NALDA Phase II. 

In 1997, due to contractor difficulties and equipment 
problems, the development team decided to convert reports to 
operate with commercial browsers, abandon X Windows and use 
Active X controls for complex reports, and move to Internet 
access. Additionally, they decided to transition from the 
IBM RISC/6000 machines to multinode IBM Scaleable 
P0WERparallel2 (SP2)' for the production platform. The SP2s 
will all be located at NAS Patuxent River Maryland, rather 
than the distributed architecture originally planned. . 
Expectations for what the system would ultimately deliver 
were scaled back. Rather than being everything for 
everyone, core capabilities were identified, and "nice to 
haves" were eliminated. Those capabilities eliminated 
included graphic capabilities, the Return on Investment 
module, and ad hoc query access against the detail data. 
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The ad hoc query capability would still be possible using 
the IQ Tool - an OLAP software product. Because of the 
detailed knowledge of the database and SQL skills required 
to make effective use of this software, the tool would be 
available only to certain users. These users would 
primarily be analysts at the major claimants or the NADEPs. 

B. THE CURRENT STATE OF THE LMDSS 

The data load into the LMDSS/NALDA II database on the 
SP2 equipment is now in its final stages. When this load is 
complete the application will be pointed to the new tables 
and turned over to the Fleet as the replacement for NALDA 
Phase I. Automated database loading software is operational 
and AV3M SALTS submissions are now being loaded directly 
into the LMDSS/NALDA II database. Naval Sea logistics 
Command (NSLC) is now out of the AV3M business (NAVAIR 
3.6.2, 1997). 

C. THE LMDSS FEATURES AND CAPABILITIES 

The LMDSS is organized into seven functional areas. 

The following descriptions of these areas and subareas have 
been derived from the LMDSS homepage: 
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1 . Management Analysis 

This module consists of various tools for displaying 
high level summary data for end items, claimants or' 
organizations. These tools identify system degraders and 
produce reports ranked by parameter. The reports include: 

• End-Item Matrix. This produces a matrix that 
summarizes reliability, supportability and cost data 
to the end-item level. 

• Claimant Matrix. This produces a matrix that 
summarizes reliability, supportability and cost data 
to the claimant level. 

• End-Item/Claiment Matrix. This produces a matrix 
that summarizes reliability, supportability and cost 
data to the end-item and claimant level. 

• Organization Matrix. This produces a matrix that 
summarizes reliability, supportability and cost data 
to the organization level. 

• Beyond Capability Maintenance (BCM) Report. This 
produces a report that summarizes BCM ddta to the 
organization level. 

2 . Candidate Identification 

This module consists of various tools for displaying 
reliability, supportability, and cost summary parameters for 



23 



selected aviation equipment. These tools identify system 
degraders and produce reports ranked by parameter. The 
primary purpose of this module is to allow managers to 
identify opportunities for life cycle cost and readiness 
improvement. These tools include: 

• Reliability/Supportability/Cost (R/S/C) Matrix. 

This offers a choice of three basic matrices: 
Component by Reported NUN, NUN Head of Family, and 
Work Unit Code (WUC) . 

• Common Equipment Matrix. This identifies potential 
problems with cross-platform components. 
Additionally, there are four methods for collection 
of equipment /systems/components that have multiple 
aircraft applicability: 

■ Local Routing Code (LRC) . This uses the ASO local 
routing code. This selection will only collect 
data for NUN or NUN Heads of family that match 
the specified LRC. 

■ Type Model. This collection method is based on 
the minimum number of Type Models in which a NUN 
must occur to be considered common equipment. 

■ Type Model Series (TMS) . This method is based on 
the minimum number of TMSs in which a NUN must 
occur to be considered common equipment. 
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■ NIIN. This selection displays the selected NUN, 
its nomenclature, the Type Equipment Code 
(TEC) /TMS it can be found on, and the number of 
the selected items in the TEC/TMS. 

• Emergent Problems Matrix. This identifies items 
that show recent changes in reliability, cost, 
maintenance and supply. 

• Bureau Number Matrix. Displays support costs, 
maintenance and supply statistics, and reliability 
figures broken out by individual bureau number for a 
TMS . 

3 . Trend Analysis 

This module consists of tools used to analyze system 
degraders to determine the basic problem (s) and examine the 
underlying cause (s). The Historical Trend Analysis tools 
display reports of statistics over time. This is tabular 
information summarized by month covering End Item or the 
component levels. 

• Aircraft Utilization History. Presents a parameter 
value entry and selection form to prepare for the 
display of flight and utilization data that aids in 
historical trend analysis of aircraft. 
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• Work Unit Code History. Presents a parameter value 
entry and selection form that includes selection of 
WUCs, to prepare for the display of maintenance data 
that aids in historical trend analysis of WUCs. 

• Intermediate Maintenance Activity. Presents a 
parameter value entry and selection form which 
includes input of NUN and Date Range to prepare for 
the display of intermediate level maintenance that 
aids in trend analysis. 

4 . Cost Analysis 

This module contains tools used to analyze end-item and 
component cost data. 

• Annual Operations and Support Costs (AIR 4.2). 
Displays platform level cost information based on 
the OPNAV Code N889 sponsored Navy Flying Hour 
Program. The report can be generated for a specific 
TMS squadron manned at 90% or 100%. 

• Budget Analysis (OP-20 Report). Displays the Budget 
Analysis Report selection parameters. T'he operator 
selects the fiscal year, TEC and funding command to 
produce the desired report. 
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• Labor Cost History. Displays Labor Cost History 
data in tabular format. Parameters of year and 
maintenance level may be selected. 

• Inflation Factors. Displays fiscal year, inflation 
rate, raw index, weighted index and budget year 
multiplier for a variety of operator selectable 
appropriations categories. 

• Item Value Cost Reports. Allows users to select 
between the Item Value to Depot Repair Cost report 
or the Item Value to Labor Cost report. In the Item 
Value to Depot Repair Cost report the user can 
compare the replacement value of items in the 
database to the cost of level 3 maintenance for a 
selected time period. The result is shown as a 
percentage showing the level 3 maintenance cost of 
in service units compared to the cost to replace 
those units. The Item Value to Labor Cost reports 
are similar in that they compare the replacement 
cost of items to the labor cost at maintenance 
levels 1 and 2 for those items. The results show 
the annual cost of ML1 and ML2 labor as a percentage 
of item replacement cost. 
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5. 



Detailed Analysis 



This module contains tools used to analyze items 
identified in the Candidate Identification Function. 

• Detailed Component Report (DCR) . This is a 
comprehensive report encompassing data from all 
three levels of maintenance (AV3M) and supply 
(Weapon System File/UICP) . It is designed to fault 
isolate candidates from the candidate identification 
to the root hardware cause (s) of R/S/C degradation. 
This is accomplished through drilling down through 
progressively more specific forms to lower levels of 
detail . 

• Supply Synopsis. This section of the Detailed 
Component Report provides greatly expanded 
information covering supply and depot repair 
parameters . 

• Source Document Report (VIDS/MAF) . This section 
provides detailed report information of VIDS/MAFs. 

It is possible to drill down to and view a specific 
VIDS/MAF. 

6 . Supply Analysis 

This module consists of specialized summary and 
forecasting reports intended for use by supply personnel. 
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This utility provides a means through which both readiness 
and cost factors are examined concurrently. 



• Wholesale System Demand. This form is used to aid 
in forecasting demand for specific NIINs. It will 
link to a NUN Analysis screen that will provide the 
user with a breakdown of more NUN specific 
information and links to the DCR and Tools Cross- 
Reference report. 

• Wholesale System Investment. This report is sorted 
by NUN and a break out of repair costs is 
displayed. 

• End-Item Material Issue Trends. This report can be 
displayed by TEC or TMS. 

• Average Customer Wait Time Reports. These reports 
can be produced for specific TECs broken out by 
maintenance level, response time crossed with COG or 
the highest wait days across all NIINs for that TEC. 

• Wait Time Maintenance Impact. Under Development 

• Average Days to Receipt. Under Development 

• Backorder History Report. Allows for the display of 
data for TEC/TMS sorted by NAVICP Material Type and 
Data Elements. The report may also include DLA 
Supply Center and Data Elements. 
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• Backorder Ranked Report. This report allows for 
identification of the NIINs with the largest number 
of backorders against them. 

• Planned vs Actual Opportunity Cost. This form 
allows the user to enter source of reliability data 
to report on from NAVICP, LSAR, or Manually Entered. 

• Mean Flight Hours Between Failures Report. This 
form and resulting report can be used for "what if" 
analysis. Planned data can be entered and then 
compared to actual data. 

• NAVICP NSN Snapshot. This report may be generated 
for a specific NIIN. Either a summary report or a 
report specific to backorders, stock status, 
alternate NIIN, etc. may be produced. 

• Repair Cycle Report. Gathers and displays data on 
repair cycle and BCM rates. 

7 . Engine Analysis 

This module consists of tools that allow the analyst to 
view projected actual costs and hours for different engines. 

• Depot Engine Repair Cost. This provides the analyst 
with historical information on the cost, funding 
source, and activity for each engine. This report 
is not accessible to contractors. 
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• Engine Overview. The overview report expands the 
information available to include engine inventory, a 
cost breakout and the capability to select data 
covering all TMSs for specific TMSs, all sites, PAC 
sites or LANT sites. 

• Engine Removal Analysis. Tracks average engine time 
since last repair when removed. The repair site and 
the number of engines attributed to that specific 
site are also listed. 

• Engine Removal Trend. Charts the engine removals to 
TMS based on 100-hour service intervals since last 
repair. Multiple series may be compared on the same 
chart, giving insight into factors such as "infant 
mortality" and "high time." 

• Top Reasons for Removal. Displays the top reasons 
for engine removals -to TMS. 

• Flight Hours Since Last Repair. This is a companion 
report to the Engine Removal Trend Report. In 
addition to charting removals by TMS, it also shows 
the reason for each removal along with flight hours 
in 100-hour increments. 

• Flight Hours Since Repair at Removal. Displays 
engine removals and maintenance man-hours. This 
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report differentiates between scheduled and 
unscheduled maintenance, and displays average hours. 

• Engine Demand Forecasting. Based on the premise 
that evolving reliability and maintainability data, 
both historically derived and imposed, must be 
considered in conjunction with historical demand for 
successful engine demand forecasting. This option 
derives from historical files and extrapolates past 
performance to future needs through application of 
relative flying hours. 

8 . Reference Information 

This module consists of tools that provide general 
aircraft information, definitions, statistics, assistance, 
reference information/reports, and information about the 
application/dat abase. 

9 . Application Management Tools 

Contains various utilities that provide current 
database status, user identification, server status, and the 
Software Change Request form. 

10 . Feature Synopsis 

This provides a brief description of the modules and 
links to the sub-elements. 
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The LMDSS has an on-line Help capability. Each input 
form has an accompanying Help function that explains the 
input fields. Additional links within some of the Help 
screens provide information on algorithms used to derive the 
reports. Other links provide definitions of acronyms and 
computer jargon. 

D. QUALITY ASSURANCE (QA) PLAN 

Prior to the transition from the NALDA I to NALDA II 
database, a systematic series of tests to ensure proper 
functionality, accuracy and a smooth transition is planned. 
The two primary targets of this testing are to assure 
accurate data retrieval throughout the application and 
database integrity. 

Sections of the application and certain derived data in 
the Oracle tables where problems have been previously 
identified are being re-coded. Concurrent with this effort, 
the QA team, composed of analysts conversant with LMDSS and 
NALDA I will go through each section of the LMDSS 
application in a critical review of form, format, and 
accuracy of content. 

The prototype application software currently in use is 
identical to that which will operate against the SP2s. This 
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means that quality assessment can begin immediately. There 
are specific challenges that must be addressed: 

1 . Data Outputs are not Identical 

Although the database which LMDSS now reaches is 
based on a data pull from NALDA I, the data outputs are not 
identical. This is because in LMDSS, the data pre- 
processing has been improved to provide greater accuracy. 
Examples of the differences include use of aircraft versions 
in addition to TMS and revised item count logic. This makes 
comparative analysis difficult, but not impossible. 

2. Software Change Requests System 

The problem is that the reporting system, which was 
built into the client-server version of LMDSS, became 
inactive during the transition to the Internet based 
version. It has been two years since a Software Change 
Request (SCR) has been classified, scheduled or answered. 
This has resulted in a lack of systematic documentation of 
problems and resolutions during the transition (Jones, 

1998 ) . 

The SCR system is again working. All SCRs and their 
status can be viewed under the Application Management Tools 
Module of the LMDSS application. An examination of the 
current list of SCRs indicates that there has been 
significant progress made on the application validation. 
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E. ANTICIPATED ADVANTAGES 

The development team anticipates that the LMDSS will 
provide an important analysis function to assist logistics 
managers in establishing requirements for new acquisitions 
as well as troubleshooting existing weapon systems. Because 
of the Integrated Data Environment (IDE), the LMDSS will 
allow all potential user levels to substantially reduce the 
amount of time required to identify and analyze problems in 
logistics support. 

The LMDSS will provide an ability to analyze data 
concerning common equipment - those items that are used on 
more than one weapons platform. The present approach 
essentially looks at present conditions and backordering 
philosophies that encourage a tunnel vision approach due to 
difficulty in identifying common components that can be used 
across platforms/weapon systems in correlating 
maintenance/repair and ordering of specific common 
equipment . 

With the implementation of the LMDSS, daily feeds of 
maintenance and flight data will be received through SALTS 
terminals directly to the production database. The current 
system uses a monthly update cycle based upon the NSLC 
processing schedule that establishes data as 45 days old as 
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the best case. Monthly reports will be available at least 
30 days earlier under the new system. Data quality will 
also be improved with the strengthening of validation 
specifications at the source and use of "most probable 
logic" within the data summarization function of the 
database. (NAVAIR 7.0, 1997) 
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IV. LITERATURE REVIEW 



A. THE DECISION SUPPORT SYSTEM AREAS OF RESEARCH 

Much of this DSS research can be classified into one of 
five areas: 

1) what distinguishes the DSS from other computer 
information technologies; 

2) whether the DSS actually improves decision quality 
or performance; 

3) identifying specific design characteristics and the 
impact they have on the DSS development; 

4) the role of the decision-maker and how differences 
between individuals and organizations can influence 
the effectiveness of the DSS; and 

5) DSS evaluation methods. 

1 . The DSS Versus Other Computer Information 
Technologies 

The first area of research has addressed what 
distinguishes the DSS from other computer information 
technologies. Currently, there is a general consensus that 
a DSS is composed of the following three interrelated 
components: data management, model management, and dialogue 
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management components (Alter, 1977; Sprague, 1980; Keen, 
1981) . Each component provides specific capabilities to a 
decision-maker and improves the effectiveness with which he 
or she works. 

The data management component should include: 

1) the capture and extraction of data into a database; 

2) the storage, retrieval, and control of data by a 
database management system; 

3) the ability to interact with data from internal and 
external sources; 

4) the ability to perform ad hoc queries; and 

5) the flexibility to allow rapid additions and 
changes in response to unanticipated user requests. 
(Alter, 1977; Sprague, 1980; Keen, 1981) 

The model management component of DSS provides a user 
with a set of capabilities that differentiate it from other 
traditional computer systems. These capabilities include: 

1) the use of multiple models to support diverse 
problems; 

2) the support of semi-structured and unstructured 
problems; 

3) the ability to build models quickly and easily; 

4) the ability to track models through a model 
directory; 
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5) the ability to integrate models with appropriate 
links through the database; and 

6) the creation, retrieval and storage of models 
handled by a model base with management functions 
analogous to database management. (Alter, 1977; 
Sprague, 1980; Keen, 1981) 

The dialogue management component provides the mode of 
interaction between the user and the DSS. Research results 
suggest that a well-developed interface should include: 

1) the support of multiple dialog styles; 

2) the capture, storage, and analysis of dialogue 
through a dialog management system; 

3) the ability to interact with the model and data 
components of the DSS; and 

4) the support of multiple methods of presenting 
output to provide for a variety of formats and 
media, and the flexibility for different users' 
knowledge base. (Alter, 1977; Sprague, 1980; Keen, 
1981) 

2 . DSS Benefits 

The second group of studies has primarily focused on 
benefits that the DSS provide. The primary justification 
for the development of a DSS is that it will be a value to 
the decision maker (Hogue and Watson, 1983) . Some studies 



39 



in this area support the premise that the DSS improves 
decision quality or effectiveness (Sprague and Watson, 1986; 
Hogue and Watson, 1985; Sprague and Carlson, 1982; Keen and 
Scott Morton, 1978). In other studies there was no effect, 
and in still others decision quality worsened when a DSS was 
employed (Benbasat and Nault, 1990; Sharda, et al., 1988). A 
DSS provides a coherent strategy for going beyond the 
traditional use of computers in structured situations where 
measures of effectiveness and efficiency are nearly 
equivalent. 6 In the semi-structured and unstructured 
situations in which the DSS is used, effectiveness has been 
the primary focus. Other research suggests that it is 
efficiency that is actually improved (Todd and Benbasat, 

1992) . 

The DSS is designed to be an interactive system used by 
managers with little experience in computers and analytical 
methods to help improve the effectiveness and productivity 
by supporting, rather than replacing, judgment (Fedorowicz 
and Manheim, 1986) . A DSS is, in effect, an assistant to 
whom the manager delegates activities involving retrieval, 

6 Efficiency is performing a given task as well as possible 
in relation to some predefined performance criterion. It is 
a measure of resources utilized against results derived. 
Effectiveness involves identifying what should be done and 
ensuring that the chosen criterion is relevant. It is the 
degree to which a goal is achieved. 
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computation, reporting, and development of alternatives. A 
manager evaluates the results and chooses the next step. 

The benefits the DSS provides are often not quantitative. 
Benefits include the ability to examine more alternatives, 
stimulate new ideas, improve confidence in the decisions, 
reduce the probability of error, improve communication of 
analysis, and speed up decision making. (Keen and Morton, 
1978; Keen, 1986; Hogue and Watson, 1985; Hogue and Watson, 
1983) 

3. Design Characteristics 

The third area of DSS research has been directed 
towards identifying specific design characteristics and the 
impact they have on the DSS development. Topics 
investigated have included the impact of different 
presentation formats, the use of color, the influence of 
different graphics capabilities, and the influence of 
different user interfaces. Analysis has generally provided 
mixed results as to the impact of these factors on decision- 
making effectiveness (Bennett, 1983; Pearson and Shim, 

1994) . This is not to infer that a DSS that is more easily 
accessible, as with a web browser, does not provide 
measurable benefits. Intuitively, the more accessible the 
design interface, the more positive the impact will be on 
decision-making effectiveness. Because internet browser 
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technology has only been introduced since the mid 1990's, 
there is a lack of research into the relative effectiveness 
of this interface versus others. 

4 . The Role of Individual Decision-Makers and 
Organ! zations 

The fourth group of studies has addressed the role of 
the decision-maker and how differences between individuals 
and organizations can influence the effectiveness of DSS. 
Research includes theoretical studies of organizational 
decision-making. Specific characteristics investigated 
include cognitive biases and processes, novice/expert 
effects, models of decision-making, and user-situational 
variables (Mittman and Moore, 1984; Mann et al., 1986; Keen 
and Morton, 1978; Alavi and Joachimsthaler, 1992). User- 
situational variables include user involvement, training and 
experience. The emphasis in this area is on describing the 
methodologies and differences of decision-making so that 
computer technologies can be effectively prescribed and 
applied to improve how decisions are made. Research has 
also found that MIS success is dependent upon the extent to 
which it fits the organizational environment (Raymond, 1990; 
Ein-Dor and Segev, 1978; Ein-Dor and Segev, 1982; Schultz 
and Slevin, 1975) . 
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5 . 



The DSS Evaluation Methods 



The last major area of research is that of measuring 
the implementation success of the DSS. Implementation 
success refers simply to realizing the intended benefits of 
the system. Currently, no single approach to the definition 
of DSS implementation success exists in the literature. A 
variety of different variables have been proposed and tested 
as indicators of success. These include such things as 
system use, decision-making time, decision-making 
performance, user satisfaction with the system, user 
confidence in the decisions, and user attitudes towards the 
DSS. (Alavi and Joachimsthaler, 1992; Money et al., 1988; 
Raymond, 1990; Lee et al., 1995; Swink, 1995; Goodhue, 1995; 
Gatian, 1994) 

The evaluation of DSS is a research direction mentioned 
by almost every author in this field, but measuring the 
effectiveness of these systems is a difficult task (Udo and 
Davis, 1992) . Again, the literature indicates little or no 
consensus as to a model or methodology for evaluating 
success. Those proposed have progressed from the 
traditional cost-benefit analysis (King and Schrems, 1978) 
to techniques that attempt to include the intangible and 
qualitative benefits of the DSS. 
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First among these is Value Analysis (Keen, 1981; Money 
et al., 1988). An important premise of this approach is 
that the perceived benefits of the DSS are significant 
determinants in justifying a specific DSS. 

Keen and Scott Morton (1978) advocate a smorgasbord 
approach to determining effectiveness. Eight methodologies 
to be matched to a specific situation are proposed. These 
include examining decision outputs; changes in decision 
process; changes in managers' concepts of the decision 
situation; procedural changes; classical cost/benefit 
analysis; service measures; managers' assessment of the 
system's value; and anecdotal evidence. 

Adelman (1992) suggests that an eclectic approach is 
required to test and evaluate DSSs effectively. He defined 
three alternative types of evaluation procedures: objective 
measurement, expert observation, and subjective judgment. 
Any one or combination of these methods could be used 
depending upon the system and what the system was to 
achieve . 

A fundamental aim of an organizational information 
system (IS) is to improve individual decision-making 
performance, and ultimately organizational effectiveness. 
The difficult in empirically assessing system effectiveness 
in this way has led researchers to adopt surrogate 
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constructs that are more easily measured. Of the two main 
approaches for evaluating IS success, the first one is 
behavioral and focuses on systems usage. This approach is 
often used in empirical research (Baroudi et al., 1986; 
Gremillion, 1984; Hogue and Watson, 1985; Raymond, 1985). 
Here the implication is that if the information system helps 
improve decision quality, then the end user will use the 
system. 

The second approach in evaluating success centers on 
user attitudes, more specifically on user satisfaction with 
various aspects of an information system (Lee, et al., 1995; 
Gatian, 1994; Swink, 1995; Hogue and Watson, 1985). End 
user IS satisfaction is the extent to which users believe 
the system meets their information requirements (Ives, et 
al., 1983). IS satisfaction is assumed to be a good 
substitute for objective determinants of information system 
success. The basic idea is that satisfied users should 
perform better than dissatisfied users and if the IS helps 
users perform better, the system is successful (Gatian, 

1994) . 

Other research, going beyond the user satisfaction with 
the’ system, has focused on satisfaction with information 
quality. Gatian' s (1994) findings support the theory that 



45 



availability of relevant information improved decision 
performance . 

Yet another method that has been proposed looks at the 
process. This methodology attempts to trace the effects of 
the system through all stages of the decision process. The 
focus is on the outcomes of the process and its individual 
steps (Vetschera and Walterscheid, 1995) . 

We believe that a combination of these methods is 
necessary to determine the success of a system. A model that 
combines the process orientation with the information 
quality methods is that of task-technology fit (Goodhue, 
1995) . The essence of this model is that task-technology 
fit is presumed to lead to higher performance. Systems that 
provide information necessary to a user's tasks, at the 
right level of detail, clearly and unambiguously will be 
highly valued. We intend to take this research one step 
further and apply it to a specific DSS. 

B. MEASURING LIFE CYCLE COSTS 

The LMDSS has been identified as a tool to reduce life 
cycle costs (LMDSS Req. Doc., 1993). In this section we 
provide a review of literature to support the significance 
of measuring life cycle cost versus alternative program 
measures . 
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The recent combination of economic trends, rising 
inflation, products and system cost growth, the continued 
reduction of buying power, and budget limitations has 
increased the awareness and interest in total system cost. 
Not only are the acquisition costs associated with new 
systems rising, but the costs of operating and maintaining 
systems already in use are increasing at alarming rates 
(NAVAIR TEAM, 1998; Hickock, 1997). The requirement to 
increase overall productivity in a resource-constrained 
environment has placed emphasis on all aspects of the system 
or product life cycle. In the past, total system cost has 
not been readily visible, particularly those costs 
associated with system operations and support. As these 
cost elements are increasingly visible through computerized 
tracking and activity-based costing, they can more readily 
be managed. Further, when addressing total cost, experience 
has shown that a major portion of the projected life-cycle 
cost for a given system or product is a result of the 
consequences of decisions made during the early phases of 
program planning and system conceptual design (Blanchard, 
1992) . Decisions at this point have a major impact on 
activities and operations in all subsequent phases of the 
life cycle. 
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Blanchard (1992) relates the cost visibility problem to 
the "iceberg effect." The acquisition cost of research, 
design, test, production, and construction are visible above 
the water. The mass of the iceberg below the surface 
illustrates additional, less visible costs such as: 

• operations cost (personnel, facilities, utilities, 
and energy) ; 

• product distribution cost (transportation, 
traffic, material handling) ; 

• software cost (operating and maintaining 
software) ; 

• maintenance cost (consumer service, supplier 
factory maintenance); 

• test and support equipment cost; 

• technical data cost; 

• supply support cost (spares, inventory, material 
support) ; 

• training cost (operator and maintenance training) ; 

• retirement and disposal cost. 

The greatest opportunity to influence total cost, which 
is predominantly made up of the costs illustrated as those 
costs below the water, is during the early phases of a 
program. Decisions relating to the evaluation of 



48 



alternative operational use profiles, maintenance and 
support policies, equipment packaging and transportation 
schemes, and level of repair concepts have a great impact on 
total cost. An overarching goal is to field high-quality 
products, systems, and structures in response to established 
needs. It is through a concurrent life-cycle approach that 
managers can deal with all economic factors (Fabrycky and 
Blanchard, 1991) and recognize the life-cycle implication 
associated with almost all decisions. Efficient and 
effective decisions result from analysis of the total 
program (cost, performance, schedule, and political 
elements) relative to the total life cycle (concept design 
and requirements planning, design and development, 
production, utilization, and retirement/disposal) . 

Product or system life-cycle analysis can be applied to the 
evaluation of numerous alternatives, including: 

1) operational scenarios and utilization approaches; 

2) system maintenance concepts and logistic support 
policies; 

3) design configurations, technology applications, 
built in test versus external test, reliability 
versus maintainability, or levels of repair versus 
discard decisions; 

4) supplier sources; 
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5) number of inventory points and levels of inventory, 
transportation and handling methods; 

6) inspection and test policies; and 

7) product recycling and disposal methods. (Fabrycky 
and Blanchard, 1991) 

We follow Fabrycky and Blanchard (1991) in holding that 
an important first step in the_ analysis is clarifying the 
analysis objectives. It is important to define the issues 
of concern and bind the problem such that the study can be 
efficient. Too large an effort can become overwhelming and 
it is easy to proceed in the wrong direction. The problem 
must be defined clearly, precisely, and presented in such a 
manner as to be easily understood by all concerned. 
Otherwise, it is unlikely an analysis of any kind will be 
meaningful. Within the established bounds and constraints, 
all possible alternatives should be considered, with the 
most likely candidates selected for further evaluation. 
Alternatives not considered cannot be adopted; therefore, it 
is better to consider all candidates even those that do not 
seem attainable or likely rather than overlook one that may 
be good. 

One of the greatest challenges facing industry, 
government agencies, the Department of Defense, and the 
general consumer of products and services is the growing 
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need for more effective and efficient management of our 
resources. The Department of Defense logistics mission is 
"to provide responsive and cost-effective support to ensure 
readiness and sustainability for the total force in both 
peace and war." (USD(A&T), 1998) The fact that logistics 
costs incurred during the operating and support phase are 
such a large part of total cost requires logistics to assume 
a major role during operational use. Given the cause-and- 
effect relationships between early planning and later costs, 
logistics has become equally significant in every phase of 
the life cycle. For these reasons research, design, 
production, logistics, and system performance analyses must 
be addressed early, concurrently performed, and integrated 
throughout the system or product life cycle. 

The above discussion demonstrates the value of measuring 
life cycle costs and the value of logistics in life cycle 
cost analysis. The challenge then becomes how to measure 
and evaluate logistics performance. Good measurements should 
cover all aspects of the process being measured, be 
appropriate for each situation, minimize measurement error, 
and be consistent with the management reward system (Menzer 
and Konrad, 1991) . Fabrycky and Blanchard (1991) recommend 
the cost breakdown structure (CBS) to provide a framework 
for defining life-cycle costs and communicating links for 
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cost reporting, analysis and cost control. CBS is a way of 
classifying costs with the classification being life-cycle 
oriented. The CBS links objectives and activities with 
resources, and sets up a logical subdivision of cost by 
functional activity area and major element of a system. It 
can be used as a basis for assessing the life-cycle cost of 
each alternative being considered. In logistics management, 
as with other management decisions, optimal solutions are 
often based on more than simply the financial bottom line. 
First and foremost systems must measure up to operational 
demands. The decisions that support those demands must also 
be fiscally responsible. 
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V. DATA COLLECTION TECHNIQUES AND 

METHODOLOGY 



A. CONDUCT OF THE STUDY 

Several techniques are commonly used to obtain 
perceptions, opinions, and judgments from subject matter 
experts. These generally fall into two categories: personal 
interviews and questionnaires. (Adelman, 1992) Both of these 
techniques were employed in the course of this study. 
Telephone interviews were conducted with logistics managers. 
These interviews were directed and structured, but allowed 
for open-ended responses in a number of specific areas. 

A structured questionnaire and copy of the 
interviewer's notes from the telephone interview were sent 
to each respondent after the telephone interview. Included 
were self addressed and stamped envelopes for the surveys to 
be mailed back. We also used follow-up e-mail and telephone 
calls in an effort to improve the response rate. 

We were also fortunate enough to be able to spend a 
week at NAVAIR. During that time, we were able to conduct 
face-to-face interviews with the PMA representatives that we 
had been unable to reach by telephone. Additionally, we 
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were afforded extensive briefings on the LMDSS capabilities, 
structure, data elements, development and history. 

B . THE SAMPLE 

We selected logistics managers as targets for our 
study. This group has been specifically identified by the 
LMDSS development team as the targeted users of the LMDSS. 
There are a total of twelve aircraft types, support 
equipment, and engine program management teams considered 
relevant to the study. 

The primary target within each PMA was the Assistant 
Program Manager for Logistics (APML) . In some cases, we 
were referred to the deputy APML, Product Support Team Leads 
or other support logisticians due to schools, retirement, 
travel or simply to provide yet another perspective. 

C . QUESTIONNAIRE DEVELOPMENT 

Our goal was to develop a questionnaire that 
ascertained the task-technology fit. In order to develop a 
questionnaire that assessed the appropriate areas a pre- 
study was conducted. We reviewed checklists and 
instructions used by logisticians, interviewed three 
logisticians and examined the LMDSS data elements. 
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The nature of this DSS evaluation was specific. 

Because we wished to elicit responses on the usefulness of 
system characteristics specific to the LMDSS, not an 
abstract system, an off-the-shelf survey instrument was not 
considered appropriate to our needs. 

A four part instrument was prepared. The first part 
was to be conducted as an interview with the logistics 
manager. This section was designed to elicit information 
about the tasks, decision environment/process, and 
information needs of the logistician. 

The next three parts were designed to be filled out by 
the respondent. Part Two was to be filled out by current 
users of the LMDSS. It called for responses in Likert-type- 
scales. See Appendix F. Questions were separated as to 
general logistics concerns and specific LMDSS queries. 
Additionally, user evaluation of specific LMDSS functions 
and data was requested. 

Part Three was designed for non-users of LMDSS. This 
section was identical to the general logistics concern 
section of Part Two. In addition, a checklist of possible 
reasons for not using the LMDSS was presented with direction 
to check all that apply. 

Part Four was applicable to both users and non-users of 
the LMDSS. This section also employed Likert-type scales to 
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elicit responses on job information needs. Listed were the 
information elements supported by the LMDSS. The respondent 
was asked to rate the usability of each element. 

Since this instrument had not been validated by 
previous research, we pretesting the survey with experienced 
logisticians. Based on their remarks, we made minor 
improvements before conducting the phone survey and then 
mailing the questionnaire to participants. The 
questionnaire is available in Appendix F. 

D. DATA ANALYSIS STRATEGY 

The survey results consisted of frequency, capacity, 
satisfaction or usability ratings assigned to 166 individual 
factors by survey participants. Data were compiled from the 
eight survey forms and entered into the Microsoft Excel 
spreadsheet program. 

For each question a frequency of total respondents 
selecting a particular rating was recorded. Trends in the 
data (that is, rank orderings) were determined instead of 
making direct comparisons of adjacent ratings, due to the 
small sample available for the survey. 
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VI . FINDINGS 



A. APML QUESTIONNAIRES 

Table VI-1 lists the APML point of contact by title and 
the method by which data was received. Of the twelve total 
aircraft, support equipment, and engine program management 
teams considered relevant to the study we were successful in 
communicating with nine. The results of the written 
questionnaire are provided in Appendix G. Seven of the 
eight written questionnaires received were from non-users of 
the LMDSS. We interviewed the E-2C APML and EA-6B APMLs but 
did not receive a completed written questionnaire. They 
both do not use the LMDSS directly, but receive reports from 
data analysts who use the LMDSS and other tools. All 
respondents are aware of the functions of the LMDSS. 

The APMLs were selected as targets for our study. This 
group has been specifically identified as the targeted 
users of the LMDSS. We were referred to additional analysts 
and logistics specialists by eight of the nine APML 
representatives interviewed as a result of non-standardized 
organization of the PMA offices and non-standardized 
logistics input. Six of nine logistics managers interviewed 
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employ civilian contractors to provide reports and analyses. 
Two of nine use NAVAIR logistics specialists. 



PMA 


Aircraft System 


Point of 


Method of Contact 






Contact 




PMA-222 


Electronic Warfare and 
Simulation, Aircraft Engine 
and Special Mission Aircraft 


NA 


Not contacted due to 
retirement of APML 


PMA- 22 5 


H-3, T-2 , A- 4 


Logistics 

Management 

Specialist 

T-2/A-4 


phone interview and 
written questionnaire 


PMA- 226 


H-46, C-130, F-4 


NA 


Not contacted - could 
not locate good phone 
number 


PMA-231 


E-2 , C-2 


APML E-2C 


Personal interview 


PMA-234 


A-6/EA-6 Intruder /Prowler 


APML EA-6B 


phone interview and 
partial written 
questionnaire 


PMA- 24 1 


F-14 Tomcat 


Deputy APML 


phone interview and 
written questionnaire 


PMA-260 


Aviation Support Equipment 


PMA 


phone interview and 
written questionnaire 


PMA-261 


C/MH-53E and Executive 
Transport Helicopter 


APML H-53 


phone interview and 
written questionnaire 


PMA-265 


A F/A-18 Hornet 


APML 


phone interview and 
written questionnaire 


PMA-276 


Light/Attack Helicopter 


Deputy APML 
for HI 
upgrades 


phone interview and 
written questionnaire 


PMA-290 


Maritime Surveillance 
Aircraft (MSA) 


Logistics 

Management 

Specialist, 

P-3 


Personal and phone 
interview; written 
questionnaire 


PMA-299 


H-2/H-60 Multi-Mission 
Helicopter 


NA 


APML away at school. 



Table VI-1 PMA Points of Contact 



Six of nine respondents work with both new aviation 
systems and sustaining existing systems. One works only 
with new aviation systems and two work only with sustaining 
existing systems. Table VI-2 shows the frequency and 
capacity respondents work with a number of tools while 
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performing their job. Table VI-3 shows the frequency and 
capacity with which respondents interface with project 
engineers, analysts, those who influence the budget, and 
others while performing their job. 

The responses of the single respondent who uses the 
LMDSS is provided in Table VI-4, Table VI-5, and Table VI-6 
that summarize the user-unique questions. The responses of 
the non-user respondents to non-user unique questions are 
summarized in Table VI-7. 

All responses are included in the data in Table VI-8 
Experience With Data Sources Other Than LMDSS, Table VI-9 
Logistics Concerns, and Table VI-10 Job Information Needs. 

B. RDBMS OR DSS? 

In Chapter IV (Lit Review) we presented a detailed DSS 
definition. A comparison of the LMDSS with this definition 
leads to the following findings: 

1 . Data Management Component 

The LMDSS fully meets these component criteria. The 
NALDA II Oracle RDBMS creates an IDE with multiple data 
sources. Ad hoc query capability - while limited to certain 
users - is available. 

2 . Model Component 

The LMDSS has no modeling capability. The LMDSS does 
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FREQUENCY OF USE 



CAPACITY OF USE 



J3 
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<Z> 
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€ 
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is 
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# 


.# 


P 

eo 

§ 


3 


Logistics Support Analysis 
(MIL-STD-1388) 


3 


4 


1 


0 


0 


0 


1 


1 


2 


Raw Data 


0 


0 


0 


i 


7 


0 


0 


1 


6 


Models 


4 


0 


0 


0 


2 


i 


0 


0 


3 


Checklists 


3 


0 


0 


0 


4 


0 


0 


0 


4 


Intuition/Experience 


0 


0 


0 


0 


8 


0 


0 


1 


7 



o 

■S 

i 



1 

1 

1 

o 

o 



Table VI-2 Logistics Management Tools 





INTERFACE FREQUENCY 




CAPACITY 
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• 














p 
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Interface with project engineers 


0 


0 


0 


0 


1 


0 


0 


1 


7 


0 


Interface with project analysts 
Interface with those who 


0 


0 


0 


0 


1 


0 


0 


1 


6 


0 


influence the budget 


0 


1 


0 


0 


0 


0 


0 


0 


4 


3 


Interface with others: 
Depot Maintenance 


0 


0 


0 


1 


3 


0 


0 


0 


4 


0 


Publication Coordinators 
Type Commanders/ 


0 


0 


0 


1 


1 


0 


0 


0 


2 


0 


Functional Wings 


0 


0 


0 


1 


2 


0 


0 


0 


3 


0 


NAVI CP 


0 


0 


0 


1 


3 


0 


0 


0 


4 


0 


Suppliers (commercial, organic) 
Integrated Logistics 


0 


0 


0 


0 


3 


0 


0 


0 


3 


0 


Support specialists 


0 


0 


0 


1 


3 


0 


0 


0 


4 


0 


Contracts Office 


0 


0 


0 


1 


0 


0 


1 


0 


0 


0 



Table VI-3 Logistics Management Interfaces 
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THE LMDSS QUERY 


FREQUENCY OF USE 


Summary data; End item 


daily 


Reliability summary parameters 


daily 


Supportability summary parameters 


daily 


Cost summary parameters 


daily 


Trend analysis (problems and causes) 


daily 


Component and/or end item cost data; 




specifically: 




Annual Operations and Support Costs 


daily 


Labor Cost History 


daily 


Item Value to Depot Repair Cost 


daily 


Budget Analysis (OP-20 Report) 


infrequently 


Inflation Factors 


infrequently 


Item Value to Labor Cost 


' infrequently 


Candidate Identification Function 




specifically: 




Detailed Component Report 


daily 


Wholesale System Demand 


infrequently 


Material Issue Trends 


daily 


Supply Synopsis 


infrequently 


Wholesale System Investment 


infrequently 


Average Customer Wait Reports 


infrequently 


Backorder History Reports 


infrequently 


NAVICP NSN Snapshot 


daily 


Mean Flight Hour Between Failure Report 


daily 


Engine Repair Cost 


infrequently 


Flight Hours Since Last Engine Repair 


infrequently 


Engine Demand Forecasting 


infrequently 


Engine Overview 


infrequently 


Reference Information, specifically: 




Aircraft Inventory and Fatigue Life 


infrequently 


Code Definition 


infrequently 


Aircraft Engine Installation and Ownership 


infrequently 


Production Load and Run Statistics 


infrequently 


Possible Courses of Action 


infrequently 


Organization Codes/Job Count 


daily 


NIIN/CAGE/Part Number Cross Reference 


daily 


TEC Information 


daily 


SALTS File Information 


infrequently 


Data Dictionary 


infrequently 



Table VI-4 The LMDSS Queries 
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FUNCTION 


RESPONSE 


Interface 


like 


Help Function 


like 


Analysis Tools 


like 


Report Presentation/Format 


like 


Time Required to get what is needed 


dislike 


Ease of getting what is needed 


dislike 


Training 


strongly dislike 


Accessibility (when desired) 


neutral 


Accessibility (server access) 


neutral 


Accessibility (password access) 


like 


Provides information needed 


like 


Table VI-5 The LMDSS 


Functions 


EXPERIENCE 


RESPONSE 


Data meet needs 


agree 


Data accessible 


agree 


Data accurate/consistent 


strongly disagree 


Data detailed enough 


agree 


The exact data meaning is clear 


disagree 


Table VI-6 Experience with 


the LMDSS Data 
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Didn't know it existed 


NUMBER OF RESPONSES 
2 


PC won't support the LMDSS 


1 


Received no training 


4 


Don’t need the information the LMDSS provides 


0 


It takes too long to get what is needed 


0 


It’s too difficult to get what is needed 


0 


It doesn’t provide the information needed 


2 


Other: Not developed for logistics (everyday) issues yet 


1 



Table VI-7 Why the LMDSS is not used 



RESPONSE 



0 } 

§ 

8 

.co 



Q) 

£ 

8 





f 


<b 

£ 

8 


1 


QJ 

$> 


n 




EXPERIENCE 


Sr 


.to 


Q) 

C 


8 


sir 


# 


Data meet needs 


0 


4 


0 


2 


8 


0 


Data accessible 


0 


1 


3 


2 


6 


2 


Data accurate/consistent 


0 


1 


3 


4 


5 


1 


Data detailed enough 


0 


2 


3 


5 


3 


1 


The exact data meaning is clear 


0 


4 


2 


3 


3 


2 



Table VI-8 Experience with Data other than the LMDSS 
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FREQUENCY OF USE 


CAPACITY OF USE 
















CO 

# 


CO 

t 










c 










o 

? 


# 




1 






<D 

8 

je 


§ 


i 




-f 


r 


o 

M 

<0 


S 


-f 

c 


LOGISTICS CONCERN 


£ 


£ 


£ 


i. 


£ 






S 


-o 


« 


Logistics criteria as input to systems design 


0 


3 


2 


0 


3 


0 


2 


i 


5 


0 


Human factors concerns 


0 


5 


1 


0 


2 


0 


1 


3 


4 


0 


Failure mode, effects, and critical analysis 
Failure reporting, analysis, and 


0 


5 


0 


1 


2 


0 


1 


2 


5 


0 


corrective-action system 


0 


2 


2 


2 


1 


1 


1 


1 


5 


1 


Provisioning needs/altematives 


0 


1 


2 


2 


2 


1 


1 


1 


5 


1 


Compatibility with existing system 


0 


2 


1 


1 


3 


0 


0 


2 


4 


1 


Configuration Management 


0 


3 


0 


2 


3 


0 


0 


2 


6 


0 


Training and training support 


0 


1 


3 


2 


2 


0 


1 


1 


6 


0 


Manpower and personnel 


0 


3 


4 


0 


1 


0 


2 


3 


3 


0 


Supply Support, spares 


0 


1 


0 


3 


4 


0 


0 


1 


7 


0 


Inventory level analysis 


0 


2 


2 


2 


2 


0 


0 


1 


6 


1 


Transportation, packaging or storage 


0 


5 


0 


2 


1 


0 


1 


1 


5 


1 


Test equipment 


0 


1 


3 


2 


2 


0 


1 


1 


6 


0 


Support equipment 


0 


1 


2 


2 


3 


0 


1 


1 


6 


0 


Computer resource support 


0 


3 


2 


2 


1 


0 


2 


1 


4 


1 


Facilities, requirements 


0 


5 


2 


1 


0 


0 


0 


2 


6 


0 


Facilities, location 


0 


6 


2 


0 


0 


0 


2 


2 


3 


1 


Data, reports requirements 
Maintenance planning 


0 


0 


1 


2 


5 


0 


0 


1 


7 


0 


(scheduled versus unscheduled) 
Level of repair analysis 


1 


2 


0 


2 


3 


0 


0 


2 


5 


0 


(0 versus 1 versus D level) 


0 


4 


0 


2 


2 


0 


0 


1 


7 


0 


Operating environment issues 


0 


3 


2 


2 


1 


0 


0 


2 


6 


0 


Cost-drivers 


0 


2 


3 


0 


3 


0 


0 - 


2 


6 


0 


Readiness degraders 


0 


2 


1 


1 


4 


0 


1 


1 


6 


0 


Cycle time to repair components 


0 


3 


2 


1 


2 


0 


0 


1 


6 


1 



Table VI- 9 Logistics Management Concerns 
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HOW USEFUL IS THE INFORMATION ELEMENT? 


JOB INFORMATION ELEMENT 


*° 


/ * 


// 
# <f 


<? 


Summary data; End item 


0 


0 


0 


2 


4 


1 


Summary data; Claimant 


0 


0 


2 


1 


2 


2 


Summary data; Organization 


0 


1 


0 


1 


5 


0 


Summary data; BCM Report 


0 


1 


0 


2 


4 


0 


Reliability summary parameters 


0 


0 


0 


1 


6 


0 


Supportability summary parameters 


0 


0 


0 


1 


5 


1 


Cost summary parameters 


0 


0 


0 


1 


6 


0 


Emerging problems 


0 


0 


0 


0 


7 


0 


Common Equipment 


0 


0 


0 


1 


4 


2 


Trend analysis (problems and causes) 


0 


0 


0 


1 


5 


1 


Component and/or end item cost data; specifically: 












Annual Operations and Support Costs 


0 


0 


0 


0 


7 


0 


Labor Cost History 


0 


0 


0 


0 


7 


0 


Item Value to Depot Repair Cost 


0 


0 


0 


0 


5 


2 


Budget Analysis (OP-20 Report) 


0 


0 


1 


2 


3 


1 


Inflation Factors 


0 


0 


1 


3 


2 


1 


Item Value to Labor Cost 


0 


0 


0 


2 


3 


2 


Candidate Identification Function; specifically: 
Detailed Component Report 


0 


0 


2 


0 


4 


1 


Wholesale System Demand 


0 


0 


1 


4 


1 


1 


Material Issue Trends 


0 


0 


1 


2 


3 


1 


Supply Synopsis 


0 


0 


1 


1 


4 


1 


Wholesale System Investment 


0 


0 


1 


3 


2 


1 


Average Customer Wait Reports 


0 


0 


1 


3 


3 


0 


Backorder History Reports 


0 


1 


1 


2 


3 


0 


NAV1CP NSN Snapshot 


0 


1 


0 


2 


4 


0 


Mean Flight Hour Between Failure Report 


0 


0 


0 


1 


5 


0 


Wait Time Maintenance Impact 


0 


0 


1 


2 


3 


1 


Average Days to Receipt 


0 


0 


1 


3 


3 


0 


Planned versus Actual Opportunity Costs 


0 


0 


1 


2 


2 


2 


Engine Repair Cost 


0 


0 


0 


0 


5 


1 


Flight Hours Since Last Engine Repair 


0 


0 


0 


1 


4 


1 


Engine Demand Forecasting 


0 


0 


1 


0 


4 


1 


Engine Overview 


0 


0 


1 


1 


2 


2 


Engine Removal Trend 


0 


0 


1 


0 


4 


1 


Flight Hours Since Engine Repair at Removal 


0 


0 


1 


0 


3 


2 


Reference Information, specifically: 
Aircraft Inventory and Fatigue Life 


0 


1 


1 


1 


3 


0 


Code Definition 


0 


0 


1 


2 


2 


1 


Aircraft Engine Installation and Ownership 


0 


1 


1 


1 


2 


1 


Production Load and Run Statistics 


0 


0 


1 


4 


1 


1 


Possible Courses of Action 


0 


0 


1 


0 


4 


2 


Organization Codes/Job Count 


0 


1 


2 


2 


2 


0 


NIIN/CAGE/Part Number Cross Reference 


0 


0 


1 


2 


4 


0 


TEC Information 


0 


0 


1 


3 


3 


0 


SALTS File Information 


1 


0 


3 


1 


0 


2 


Data Dictionary 


0 


0 


0 


4 


1 


2 



Table VI-10 Logistics Management Information Elements 
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offer a module entitled Trend Analysis. This module 
provides historical data in tabular format. Historical data 
is also presented in a format that could support time-series 
forecasting. This is found in Engine Demand Forecasting in 
the Engine Analysis module, and Wholesale System Demand in 
the Supply Analysis module. Also within the Supply Analysis 
module are two subareas that can accept manual entries in 
some parameters. This allows for a degree of "what if" 
analysis for Mean Flight Hour Between Failures Report and 
Planned vs Actual Opportunity Cost. However, there is no 
sensitivity analysis capability available to support the 
"what if" analysis. 

3 . Dialogue Management Component 

The LMDSS meets, to some degree, all of the criteria of 
this component. The browser interface and hyperlinks offer 
navigational flexibility. Usage of the OLAP tool provides 
an additional degree of flexibility. Output either can be 
to the screen in HTML format or can be e-mailed to the 
requester in a format that can be imported to a spreadsheet 
application. There is no graphics capability. 

C . DATA QUALITY 

In the course of this study, three data quality issues 
were identified. 
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1 . Data Accessibility 

The IDE improves accessibility of data. The 
LMDSS/NALDA II database reduces the necessity of querying 
multiple disparate databases to retrieve relevant data for 
analysis. An economic analysis undertaken as part of the 
NALDA II Milestone III approval process found that the new 
system will allow all potential users to substantially 
reduce the amount of time required to identify and analyze 
problems in logistics support by incorporating data from as 
many as nine other disparate databases which are currently 
used for analysis. A cost avoidance of $2 million is 
expected over the life cycle of the LMDSS. In the area of 
Common Equipment Analysis, there is a potential cost 
avoidance of $19 million. The IDE allows for performance 
and reliability of specific components across the whole 
spectrum of Naval aircraft to be ascertained. This data 
accessibility did not exist prior to the LMDSS. (NAVAIR 7.0, 
1997) 

There is, however, misunderstanding on the part of many 
logistics representatives of this data accessibility. A 
perception exists that the LMDSS and NALDA II will hamper or 
eliminate the analyst's ability to access detail data (NAWC 
AD 3. OB, 1998). The Support Equipment PMA believed that the 
Cost Analysis capability would be inadequate for their needs 
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because cost data from DLA was not included. A review of 
the external interface data for NALDA II indicates that 
extensive DLA cost data will be available (Capstone, 1997) . 

2 . Data Consistency 

Data consistency has two aspects. Consistency between 
data retrieved under NALDA I and NALDA II, and consistency 
between modules in the LMDSS. 

Although the database which the LMDSS now reaches - 
NALDA II - is based on a data pull from NALDA I, the data 
outputs are not identical. This is because in the LMDSS, 
the data pre-processing has been improved to provide greater 
accuracy. Examples of the differences include use of 
aircraft versions in addition to TMS and revised item count 
logic. 

Currently, there are identified inconsistencies between 
the LMDSS modules. These are being addressed through the on 
going quality assurance process (Jones, 1998; SCR Sub- 
module, LMDSS application) . 

3 . Data Validity 

Data validity was a concern expressed by 75% of survey 
respondents and interviewees. The general perception was 
that data validity was currently poor and would continue to 
be poor. 



68 



One example that was offered by the SE PMA to 
illustrate this issue related to Maintenance Level 1 
(organizational level) , Maintenance Action Forms (MAFs) for 
tow tractors. 

• 47% of MAFs recording down time due to Awaiting 
parts had no failed parts documented 

• 32% of MAFs attributed the failed part to the part 
number of the tow tractor 

• 272 MAFs recorded removal and replacement of the tow 
tractor part number. 

Problems like this arise, in part, because finding the 
correct work unit code (WUC) or part number can be a time 
consuming task. Busy maintainers memorize a few key WUCs 
and part numbers and use those regardless of the real 
discrepancy. Lack of training on the importance of data 
validity may also contribute. 

Another example was offered by the logistics management 
specialist for the S-3 aircraft. A data query to identify 
readiness degraders resulted in identifying the airframe as 
the top system degrader. Further investigation showed this 
was a result of how scheduled maintenance washes were being 
documented in an aircraft squadron. Additional stories were 
prevalent of the adverse impact of the use of inaccurate 
type equipment codes (TEC) or WUC on MAFs by maintenance 
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technicians who do not understand how the data is used. 
Seasoned data analysts know where to look to find erroneous 
data and know how to remove misleading data from logistics 
management reports. These data validity problems are caused 
by poor documentation at the source. This problem is 
currently being addressed by improved Validation 
Specifications at the input point of NALCOMIS. 

A second data validity area is cost data. The 
maintenance cost data is incomplete. Maintenance Level 3 
(depot) was described during one interview as a "black 
hole." The only aircraft cost data for ML3 in the database 
is aggregate cost. It is not broken out by TMS . Engine 
overhaul cost data - what ML3 charges the fleet - is 
available by TMS. The LMDSS database has placeholders for 
detailed cost data, but that data is currently unavailable 
from the depots. This precludes total cost visibility. 

A final area of concern with data validity is the new 
SALTS processing procedures. When NSLC had cognizance of the 
SALTS data, a "scrubbing" process was used. Five areas of 
the detail data received from the fleet activities were 
compared with a 79 Record. If there were a 10% or greater 
discrepancy all detail data from that activity would be 
rejected. 
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Under the new system, the data will not be scrubbed 
prior to being loaded into the database. The rational for 
this change is two fold. First is the belief that there is 
more value to the analysts in having all of the data even if 
a small percentage of them are in error. Under the old 
processing system, when data were rejected the fleet 
activity had to correct the data and then resubmit. It 
could take up to three months before the detail data was 
integrated into the database. Second is that the Most 
Probable Logic feature used when summarizing the detail data 
will identify and correct these errors. 
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VII. DISCUSSION 



A. APMLS AS USERS 

We selected the APMLs for our study because this group 
has been specifically identified as the targeted users of 
the LMDSS. Of the twelve total aircraft, support equipment, 
and engine program management teams 'considered relevant to 
the study we received seven complete responses and two 
partial responses. The partial responses were a phone and a 
personal interview without completion of the written 
questionnaire. Additionally, we conducted personal and 
phone interviews with logistics advisors and the LMDSS 
program representatives. 

Our data collection was limited by two constraints. 

The LMDSS is a prototype system and is not yet widely used, 
and the APML is only one of many potential users we 
subsequently identified during the course of our research. 
Because the tasks and needs of different users vary greatly, 
the information derived from the study of APMLs cannot be 
used to interpret the needs of other population groups 
without considerable risk. We identified the following user 
groups as equally important as AMPLs to logistics input to 
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aviation program management decisions: IPT data analysts, 

FST data analysts, and Type Commands data analysts. These 
analysts may be Navy data analysts, government employees, or 
civilian contractors assigned within the teams. 

As previously discussed, program management and the 
logistics input thereto are not standardized. Each PMA 
determines how he will manage his program. The organization 
may primarily be within the program office, the aircraft 
controlling custodian, or the depot maintenance engineering 
support organizations. Additionally, program offices use a 
number of different sources for logistics support. Some 
program managers rely heavily on Navy personnel assigned 
within the program office, others rely on government 
employees from a logistics competency group in NAVAIR, while 
some contract out to commercial sources for logistics 
analyses and recommendations. Furthermore, PMAs must 
interface with support environments such as policy, process, 
facility and infrastructure organizations to develop optimal 
policies and processes; report actions taken and results; 
and obtain fleet feedback on system performance. All of 
these activities point to additional users of naval aviation 
logistics data and the LMDSS . 

When describing the APML job, respondents commonly 
referred to designing, developing, or analyzing support 
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infrastructures for aircraft and aircraft systems. LTCOL 
Wiechowski, H-53 APML, referred to this job task as the 
"care and feeding" of the heavy lift helos. The support 
infrastructure is not limited to logistics concerns. As 
presented in Table VI-3, logistics managers interface 
frequently (daily or weekly) with project engineers, project 
analysts, those that influence the program budget, and 
others to both identify and analyze requirements. 

B. THE LMDSS USE 

As presented in Table VI-7, the following reasons were 
most frequently cited for why the LMDSS is not used. 

• Received no training - 40% 

• Didn't know it existed - 20% 

• It doesn't provide the information needed - 20% 
Training is a critical issue. Many studies of information 
technology deployment in organizations have found that 
failure of these systems can be attributed to the lack of 
relevant and satisfactory training programs provided for all 
levels of end users. (Lee, et al, 1995; Nelson and Cheney, 
1991; Udo and Davis, 1992) Training is an on-going effort by 
the NAVAIR training team. In addition to conducting 
training for data analysts located at Patuxent River, site 
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visits to users located at other bases have been and are 
being conducted. 

NAVAIR 3.6.2 continues to advertise the coming of the 
LMDSS. This is accomplished through the web-site and 
frequent presentations on the state of the application. As 
the training team continues to reach more potential users, 
awareness of its existence will increase. 

As pointed out in the Findings Chapter there is a basic 
discontinuity between the survey respondents' perception of 
the data available under LMDSS/NALDA II and what will 
actually be available. When the LMDSS and NALDA II are 
fully on-line, all of the data available with NALDA I will 
be accessible. 

The way that the LMDSS has been advertised on its Web 
page since becoming available via the Web has contributed to 
this confusion. The LMDSS Web page does not identify the 
application as being a prototype. It does not identify the 
database as being not fully loaded. This has led some 
respondents to believe that what they currently see is what 
they will ultimately get. In actuality, the stated 
information needs from the questionnaires are provided by 
the LMDSS as discussed in part D of this chapter. 

It is important to overcome this perception. End users 
need to regard the information systems they are using and 
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the information provided by the IS as relevant and useful 
for their job performance, if they are to accept such 
systems. (Lee, et al., 1995; Gatian, 1994) 

In the domain of DSS, the fundamental role of computer 
support is to assist people in reaching decisions about the 
course of action to implement in a particular problem 
situation. A user must reach a cognitive state where he 
understands the issues sufficiently well to choose to act or 
not. The computer tools must be designed to provide this 
fundamental layer of support to the user. The user has many 
things at stake - position, reputation, and self-image - and 
as such will rarely be willing to treat the computer as a 
black box, which tells him what course of action to 
implement. (Fedorowicz and Manheim, 1986) With this in mind, 
the issues of data validity and data summarization/origin 
must be addressed. 

While the distrust of the data validity is not unique 
to the LMDSS or NALDA II, it is a real as well as perceived 
problem. Additional training at the data input source, 
continued development of NALCOMIS Validation Specifications, 
and pursuit of Automated Maintenance Environment (AME) 
initiatives are all possible means of improving data 
validity. 
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A thorough understanding of the origin and derivation 
of data is necessary if users are to trust and fully utilize 
the data resource (Brackett, 1996) . With the exception of 
the Cost Analysis module, the algorithms for deriving the 
data and the data sources are not explicit. There is an on- 
line data dictionary available, but in our opinion, it adds 
little or no clarity to the subject. 

The lack of consistency between outputs under NALDA I 
and LMDSS/NALDA II should also be explained to the users. 
Preprocessing, NALCOMIS Validation Specifications, and the 
use of Most Probable Logic have all improved data accuracy, 
but in doing so created differences between outputs. The 
sources of these differences need to be explicit to the 
user. 

When people understand the content and meaning of all 
data, the use of those data to support current and future 
decision needs is limited only by people's imagination. 
Improve the quality of information relative to timing, 
accuracy, relevancy, objectivity and understandability and 
the quality of the resultant decision making should be 
improved. (Stephenson, 1986; Gatian, 1994) 
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C. LOGISTICS CONCERNS 

The logistics managers identified the following as the 
logistics concerns they work with most frequently (five or 
more respondents work with them weekly or daily) . 

• Data, reports requirements - 87% 

• Supply support, spares - 87% 

• Maintenance planning - 63% 

• Readiness degraders - 63% 

• Configuration Management - 63% 

• Support equipment - 63% 

Technical data is one of the four areas identified to reduce 
costs as presented in the Affordable Readiness Proposed 
Metrics. Spares and support equipment are elements of 
inventory, which is identified as another area to reduce 
costs. Readiness degrader analysis is central to 
reliability-based logistics 'and trigger-based item 
management that are used to implement the sustainment 
element of Affordable Readiness, (see Appendix A) 

Supply support, configuration management, support 
equipment, maintenance planning, and level of repair 
analysis are all specifically addressed as elements of total 
cost of ownership, maintenance concept, standardization, and 
supportability . These four factors are identified by 
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ASN (RD&A) as those that must be considered to sustain 
support and reduce costs. (see Appendix C) 

Logistics managers identified the following logistic 
concerns as those they work with least frequently (five or 
more respondents never or infrequently work with them) . 

• Facilities, location - 75% 

• Facilities, requirements - 63% 

• Transportation, packaging or storage - 63% 

• Failure mode, effects and criticality analysis - 63% 

• Human factors concerns - 63% 

Intuitively, the location of facilities is of little concern 
to logistics managers because there is little opportunity to 
influence this decision. Examining the influences 
surrounding military base closures and realignments gives us 
a context in which to appreciate the limitations of an input 
by an individual APML or PMA to influence this factor. 
Additionally, we understand why logistics managers 
infrequently work with facility requirements because this is 
only a concern during system acquisition, upgrade, or 
relocation. It is not a concern that impacts every stage of 
the life cycle, as other concerns do. 

Specific program transportation and handling 
requirements are derived from the maintenance concept and 
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standardization factors (Blanchard, 1992) . The Navy has a 
mature and responsive transportation, distribution and 
storage system. Logistics managers are more concerned with 
the maintenance concept concerns (maintenance planning, 
level of repair analysis, local repair versus transport to 
repair and return) and standardization concerns 
(configuration management and interchangeability) that 
contribute to transportation, packaging and storage 
concerns. The Navy transport and storage system imposes 
constraints (weight limits, cubic space limits, hoist point 
or shock requirements) within which the APML must work. We 
understand why a logistics manager will focus on the 
contributing concerns rather than fixed constraints. 

Failure mode, effects, and criticality analysis (FMECA) 
is a design tool and analysis method used to tailor the 
complexity of the design, identify possible system failures, 
the causes of these failures, the effects of the failure on 
the system, and the criticality in terms of safety and 
mission accomplishment (Blanchard, 1992). A logistician who 
is involved with the early development of a system will work 
with this concern, but logisticians who do not get a voice 
in design will not, nor will those who are working with 
sustaining existing systems. 
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The objective of human factors analysis is to assure 
compatibility between the system physical and functional 
design features and the human element in the operation, 
maintenance, and support of the system. Human factor 
analysis is an integral part of overall system analysis. 
Operator and maintenance personnel requirements, and 
training program needs evolve from an iterative process of 
evaluation, system modification, and reevaluation. 
(Blanchard, 1992) Human factors concerns are more directly 
associated with engineering design and performance analysis 
efforts. As logistic concerns become more integrated with 
engineering design and performance concerns we can expect 
human factors requirements and criteria will increase in 
importance . 

D. JOB INFORMATION NEEDS 

Table VII-1 lists information elements indicated to be 
the most useful in performing the APML job. All of the 
elements presented in the questionnaire were useful to some 
degree to someone and there was no element added by a 
respondent. The elements included in the questionnaire but 
not listed in the table, are those that the respondents did 
not consistently indicate as either slightly useful or 
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extremely useful (five or more respondents; four or more for 
engine elements) . 

The LMDSS provides information for each of the 
identified elements except Candidate Identification: Wait 
Time Maintenance Impact and Average Days to Receipt which 
are under development. As discussed in the Data Quality 
section of Chapter VI, the only aircraft cost data for 
Maintenance Level 3 (depot) in the database is aggregate 
cost. It is not broken out by TMS other than engine 
overhaul cost data. The Reference Information : Production 
Load and Run Statistics element provides information on the 
LMDSS not aviation programs. Respondents indicated the 
element is useful, but we believe the question may have 
falsely led them to believe it was for aviation programs not 
the LMDSS. The respondents did not use the LMDSS, this 
reference element was listed among aviation system reference 
elements, and respondents were not asked to clarify one 
reference information element from the other. Engine 
Information elements did not pertain to the response 
received from the Support Equipment APML. 

The LMDSS Trend Analysis module is count-based and does 
not include graphics capabilities. To perform a regression 
analysis or analysis of possible cause and effect 
relationships the LMDSS data must be further manipulated 
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Summary Data; End Item 
Summary Data; Organization 

Summary Data; Beyond Capability Maintenance 

Reliability Summary Parameters 

Supportability Summary Parameters 

Cost Summary Parameters 

Emerging Problems 

Common Equipment 

Trend Analysis (Problems and Causes) 
Component and/or End Item Cost Data, 
Component and/or End Item Cost Data, 
Component and/or End Item Cost Data, 
Component and/or End Item Cost Data, 
Component and/or End Item Cost Data, 
Component and/or Ene Item Cost Data, 
Candidate Identification Function, 

Candidate Identification Function, 

Candidate Identification Function, 

Candidate Identification Function, 

Candidate Identification Function, 

Candidate Identification Function, 

Candidate Identification Function, 

Candidate Identification Function, 



(BCM) Report 



specifically: 
specifically : 
specifically: 
specifically: 
specifically: 
specifically : 
specifically : 
specifically: 
specifically: 
specifically : 
specifically: 
specifically: 
specifically: 
specifically : 



Annual Operations and Support Costs 
Labor Cost History 
Item Value to Depot Repair Cost 
Budget Analysis (OP-20 Report) 
Inflation Factors 
Item Value to Labor Cost 
Wholesale System Demand 
Material Issue Trends 
Supply Synopsis 
Wholesale System Investment 
Average Customer Wait Reports 
Backorder History Reports 
NAVI CP NSN Snapshot 



Mean Flight Hour Time Between Failure (MTBF) 

Candidate Identification Function, specifically : 

Candidate Identification Function, specifically: 

Engine Repair Cost 
Flight Hours Since Last Engine Repair 
Engine Demand Forecasting 
Engine Removal Trend 

Reference Information: Aircraft Inventory and Fatigue Life 

Code Definition 

Production Load and Run Statistics* 
Possible Courses of Action 
NIIN/CAGE/PN Cross Reference 
TEC information 
Data Dictionary 



Report 

Wait Time Maintenance Impact 
Average Days to Receipt * 



Reference Information: 
Reference Information: 
Reference Information: 
Reference Information: 
Reference Information: 
Reference Information: 



notes : 

Engine information elements do not pertain to Support Equipment APML. 
Aircraft Fatigue Life Reference Information does not pertain to all aircraft 
* indicates the LMDSS function is under development 
** provides reference information for the LMDSS not PMA program 



Table VII-1 Most Useful Information Elements 



beyond the LMDSS application. We believe these shortcomings 
reduce the utility of the Trend Analysis function. The 
Possible Courses of Action area simply includes a checklist 
of actions to consider and is not tailored to a specific 
program or system, forecast, or available data. 
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E. 



AFFORDABLE READINESS DECISIONS 



The proposed Affordable Readiness metrics discussed in 
Chapter II reflect efforts to more accurately capture the 
total cost of ownership of aviation systems. As we have 
discussed, cost reductions are considered in conjunction 
with support, readiness and safety considerations. The 
LMDSS is designed to be a tool to facilitate continuous 
action by NAVAIR logistics management teams to measurable 
reduce the life cycle support costs (or the total cost of 
ownership) of aviation systems while protecting readiness. 

To measurably reduce the associated costs, the LMDSS must be 
able to measure the associated costs. The metrics proposed 
by Affordable Readiness define the required measurements. 

The current architecture and capability of the LMDSS as a 
NALDA Phase II application adequately support measurement of 
the metrics associated with all proposed areas of Affordable 
Readiness except the metrics associated with safety (Class A 
mishaps) . 

The reduction in the life cycle support costs of 
aviation systems will take more than the ability to measure 
associated costs. In addition to knowing what the costs are 
one must be able to analyze why and have incentives to make 
the "right" decisions. The LMDSS has the ability to provide 
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useful data as defined by Affordable Readiness metrics. In 
addition to providing data, users must have confidence in 
those data. Our research did not include a comparison of the 
LMDSS analysis capabilities versus alternative methods of 
analyzing data. The perception of APMLs is the LMDSS is 
good at "big picture" and indicating "where to go look" but 
falls short of communicating details with ease or indicating 
"why" a system measurement is as it is (such as what is 
degrading mission capability or why a component is failing) . 

Naming an application a DSS creates certain 
expectations. One of those expectations is that the DSS 
will support all three phases - intelligence, design and 
choice - of the decision making process. In order for a DSS 
to support the intelligence phase, it must provide accurate, 
timely information. The design phase includes inventing, 
developing and analyzing possible courses of action. The 
analysis capability is fulfilled by being able to answer 
"what if" questions. The ability to suggest new 
alternatives is met by being able to perform goal seeking. 
The choice phase involves assistance in the selection of the 
alternative to be implemented. Generally, this is 
accomplished with an optimization routine. 

The LMDSS fully supports the intelligence phase of the 
decision making process. In order to support the other 
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phases of the decision making process, the data must be 
exported to other applications that offer models or 
forecasting utilities. As one survey respondent commented, 
"The LMDSS can help answer the what, but it can't help me 
with the why or the how." 

The LMDSS provides the facility to export data to 
other applications. But, if the LMDSS is to fulfill its 
stated purposes of providing a repeatable decision making 
process it should offer a standardized set of modeling and 
forecasting tools as part of the LMDSS application. 

F. PROGRAM MANAGEMENT ENVIRONMENT 

Because the LMDSS will not be introduced and cannot be 
evaluated in isolation, we briefly address the program 
management environment and culture to more fully complete 
the context of our analysis. 

The LMDSS is designed to support APMLs and in turn to 
benefit PMAs . Navy acquisition and program management is 
tied closely to the planning, programming, and budgeting 
(PPBS) process which determines which DoD requirements get 
funded and which do not. Those programs that get funding 
survive. Those that do not perish. A GAO report of 
December 1996 made the following recommendations to improve 
opportunities to enhance DoD' s Logistics Strategic Plan: 



87 



To build on DoD’s existing strategic planning efforts and to have a better 
chance of achieving the major logistics system improvements that its plan 
envisions, we recommend that the Secretary of Defense direct the Deputy 
Under Secretary for Logistics to (1) ensure that future logistics plans 
include a recognition of the magnitude of the investment that is required to 
accomplish the plan’s goals, objectives, and strategies and (2) issue 
guidance to the Secretaries of the Army, the Navy, and the Air Force and 
the Director of DLA instructing the services and DLA on how to link their 
goals and budgets to the DoD logistic strategic plan’s overall goals and 
strategies. (GAO/NSIAD-97-28, 1996) 

As indicated, change to the logistics processes must 
compliment the organizational structure (i.e. be linked to 
overarching Navy goals) and adequate resources must be 
dedicated to turn strategy to action. The change must also 
consider other organizational factors such as measurements, 
control processes, and reward systems. Acquisition Reform 
initiatives direct PMAs to tailor programs, be more 
creative, and to consider the total cost of ownership 
(DoDInst 5000.2; Hickok, 1997; Fox 1997). Contrary to these 
directives, the predominant focus of program management 
remains on unit cost, schedule, and design performance (Fox, 
1997; Eaton, 1997) . Incentives continue to support driving 
down acquisition cost (unit cost), ensuring timely delivery 
of aviation systems (schedule) , and meeting performance 
specifications. To measurably reduce life-cycle support 
costs PMAs need more than a tool with which to measure them. 
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As Kaminski, then USD(A&T), said when questioned what he saw 
as the major improvements yet to be achieved in acquisition 
reform: 

Probably the biggest one is really being serious about addressing life-cycle 
cost. That is an area that I think we still talk about today, but I do not 
think we have followed through with serious initiatives. I still do not 
believe we have sufficient incentives in place for most program managers 
to seriously consider the life-cycle costs of their program... The incentives 
are still too much in the direction of saving near-year monies, and that 
support costs will be someone else’s problem in the out-years. (Fox, 1997) 

Kaminski tied incentive problems to the budget process. 
A program manager has to put up near-year funds (taken from 
another program areas) to make improvements and then when 
the out-year savings are realized those funds are swept away 
and are not available to the program. (Fox, 1997) 

A logistics analyst for the S-3 aircraft annually 
updates a list of Logistics Engineering Change Proposals 
(LECPs) that has initiatives from ten years ago. The list 
documents projected total cost savings to by proposed 
investments in engineering changes. The list is kept from 
year to year because the proposals remain unfunded. Scarce 
O&S funds continue to be spent to maintain systems that 
cannot be upgraded without investment that require 
procurement funds. The LMDSS may help identify and justify 
LECPs, but it will not remedy these types of problems 
driving up life-cycle costs. 
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Additionally, while PMAs are encouraged to be creative, 
they are discouraged from taking risks. In fact, they are 
expected to plan carefully to manage and mitigate risk 
(DoDInst 5000.2; Conrow and Fredrickson, 1996; Rudwick, 

1992) . DoD/DoN strategies recognize the need to change 
culture and well as impediments to do so (Fox, 1997; Hickok, 
1997; GAO/AIMD-96-109, 1996; GAO/NSIAD-95-28, 1994; 
GAO/AIMD/NSIAD-94-101 ) . The LMDSS implementation must 
consider DoD/DoN strategies, environment, culture, and other 
organizational factors. 
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VIII. CONCLUSIONS AND RECOMMENDATIONS 



The data and information collected for this study on the 
effectiveness of the LMDSS to support logistic management 
decision-making provides ample material to draw conclusions 
pertinent to this study and identify areas that warrant 
further research. 

A. CONCLUSIONS 

1. The LMDSS is not a Decision Support System. 

A DSS is composed of the following three interrelated 
components: data management, dialog management and model 
management. The LMDSS fully meets the data management 
component criteria. It meets, to some degree, all of the 
dialog management component criteria. The LMDSS has no 
modeling or sensitivity analysis capability. The LMDSS 
Trend Analysis module provides historical data in tabular 
format. Historical data is presented in a format that could 
support time-series forecasting, but not causal, and there 
is limited "what if" analysis capability. It is a 
relational database that improves data accessibility. 

2 . There are multiple user groups who will be users 
of the LMDSS. 
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User groups include IPT data analysts, FST data 
analysts, and Type Command data analysts. These analysts 
may be Navy data analysts, government employees or civilian 
contractors assigned within the teams. 

3 . The LMDSS meets information needs to implement 
Affordable Readiness initiatives . 

The current architecture and capabilities of the LMDSS 
provide information and statistics associated with all 
proposed logistics management areas of Affordable Readiness. 
No additional information needs were identified by surveyed 
respondents. Lack of graphics, modeling and sensitivity 
analysis capabilities limit identification, analysis and 
comparison of Affordable Readiness initiatives. 

4. Data quality is both a real and perceived problem. 

We identified the following three data quality issues: 

accessibility, consistency and validity. The LMDSS improves 
the accessibility of data with the IDE. However, a 
perception exists that it will hamper or eliminate the 
analyst' s ability to access detail data. Data consistency 
is adequately addressed through the LMDSS Quality Assurance 
process. Poor documentation at the source degrades data 
validity and the lack of Maintenance Level 3 (depot) cost 
data precludes total cost visibility. 
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5 . 



The LMDSS effectiveness in measurably reducing 



life-cycle support costs is hampered by the environment in 
which aviation program management decisions are made. 

The LMDSS has the capability to support decisions to 
reduce life-cycle support costs, but PMAs need more than a 
tool with which to measure life-cycle costs to reduce them. 
Incentives continue to support driving down acquisition cost 
(unit cost), ensuring timely delivery of aviation systems 
(schedule), and meeting performance specifications. The 
current environment encourages short-term decisions that 
compromise life-cycle decisions. The LMDSS can help 
identify and justify decisions to reduce life-cycle costs, 
but other factors are driving up these same costs. 

B . RECOMMENDATIONS 

1 . Incorporate a standardized set of modeling tools 
and sensitivity analysis as part of the LMDSS application. 

To fully support the decision-making process, modeling 
capabilities are necessary. Providing a standardized set of 
modeling tools will ensure comparable analysis and 
comparison across aviation systems. Sensitivity analysis 
capabilities would allow analysts to more readily assess the 
impact of different decisions. 
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2 . 



Incorporate graphics capability as part of the 



LMDSS application. 

Currently, data output is available in tabular format 
only. A DSS should support multiple methods of presenting 
output. This would add flexibility to support different 
users' knowledge bases. 

3 . Enhance availability of algorithm and data source/ 
summarization documentation. 

A thorough understanding of the origin and derivation 
of data is necessary if users are to trust and fully use the 
data resource. Adding specific data source and algorithm 
information to the data dictionary is warranted. 

4. Expedite initiatives to improve data validity. 

Data validity problems are not unique to the LMDSS and 

NALDA II. The Most Probable Logic function used in the data 
summarization improves the validity of summarized data, but 
poor documentation at the source precludes valid detail 
data. Initiatives, such as Automated Maintenance 
Environment (AME) and Optimized NALCOMIS are crucial to 
meaningful improvements in data quality. 

5 . Collect and provide Maintenance Level 3 (depot) 
detail cost data. 
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Lack of detail cost data from ML 3 precludes total cost 
visibility. Total Cost visibility is fundamental to making 
intelligent life-cycle cost decisions. Although placeholders 
exist in the LMDSS database, they cannot be used until ML 3 
collects and provides this data. 

6. Align Budget Process, Reward Structure, and 
Strategic Decision efforts to support life-cycle cost 
reduction initiatives . 

Until the entire decision-making environment is aligned 
around a common goal of reducing life-cycle costs, efforts 
in this area will be fragmented and undermined by short-term 
imperatives. Program Managers must be effective advocates 
of total cost of ownership. In order to accomplish this, 
they must be encouraged to take risks and be creative when 
considering life cycle costs and they must be rewarded for 
doing so. 

C. RECOMMENDATIONS FOR FURTHER RESEARCH 

1 . Evaluate modeling tools currently being used by 
logistics management teams and commercial modeling tools 
currently available. 

A comparison, analysis, and identification of the best 
set of standardized modeling tools will benefit efforts to 



95 



incorporate modeling capabilities to meet logistics 
management decision-making needs. 

2 . Evaluate graphics capabilities currently being 
used by logistics management teams and commercial graphics 
tools currently available. 

A comparison, analysis, and identification of the best 
set of graphics tools will benefit efforts to incorporate 
modeling capabilities to meet logistics management decision- 
making needs. 

3 . Conduct a survey of the newly identified users 
once the LMDSS is a production system. 

We selected the APMLs as targets for our study. 
Additional users were identified. Because the tasks and 
needs of different user groups vary, the information derived 
from this study of APMLs may not adequately transfer. Our 
study was also constrained by the fact that the LMDSS is 
still a prototype system. Evaluating the capability of the 
production system to meet user needs is warranted. 

4. Conduct a study of data validity. 

During the course of this study we identified some 
areas where data validity problems exist. Further research 
is warranted to analyze additional data validity problem 
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areas, assess the impact, and evaluate alternative courses 
of action. 

5 . Assess the readiness for change and develop an 
organizational transition plan for implementing total cost 
of ownership initiatives . 

NAVAIR is currently attempting to change the focus from 
readiness at any cost to Affordable Readiness. Effective 
transition from one state to another is unlikely unless 
there is an adequate perceived need for change, the 
organization structure, reward system and processes are in 
place to support that change, and the change is effectively 
managed. A study of where NAVAIR is in the process, where 
they are going and how best to get there is warranted. 
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APPENDIX A: AFFORDABLE READINESS 



Affordable Readiness is a business practice with four inter- 
related elements: flexible sustainment, sustained 
maintenance planning, rightsourcing, and total cost of 
ownership. 

Flexible sustainment encourages program managers to use 
performance-based specifications; develop innovative, cost 
effective, life-cycle solutions; conduct supportability 
analyses; and improve reliability. It is implemented 
through reliability-based logistics and trigger-based item 
management . 

Sustained maintenance planning initiatives include 
reliability improvements; cycle time reductions; process 
improvements; technology insertions; and infrastructure 
improvements . 

Rightsourcing is defined as "selecting the most 
advantageous source to accomplish a specific function for a 
weapon system in its life cycle. Selection criteria 
include, but are not limited to life cycle cost, quality, 
reliability, safety, and effect on other programs. Specific 
functions may include all facets of Design, Production, 
Operation, Logistics Support, and Disposal of the system." 
(NAVAIR, 1998) 

Total ownership costs include all costs associated with 
the research, development, procurement, operation, 
logistical support and disposal of an individual weapon 
system and the related infrastructure. 

For additional readings on Affordable Readiness, 
reliability-based logistics, and trigger-based management 
see www.nalda.navy.mil, NAVAIR Logistics, Affordable 
Readiness Link. 
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APPENDIX B: INTEGRATED PRODUCT TEAM 



In the context of a DoD acquisition program there are three 
types of IPTs: overarching IPT, working-level IPT and 
Program IPT. 

The overarching IPT is formed for each program to 
provide assistance, and oversight as the program proceeds 
through the acquisition life cycle. It is composed of the 
PMA, Program Executive Officer (PEO), and appropriate 
component staff, joint staff and Office of the Secretary of 
Defense staff principals or their representatives. 

Working-level IPTs are composed of the PMA or his 
representative, and the appropriate staff members who can 
assist the program by providing functional knowledge and 
expertise to the program. For major programs working-level 
IPTs are generally focused on a particular discipline or 
functional area such as supportability, testing, 
cost/performance or contracting. For smaller projects one 
working-level IPT may be focused on the entire effort. The 
integrated IPT is an exception to this rule. The PMA may 
establish an integrated IPT to coordinate the activities of 
the other working-level IPTs. Ideally, the integrating IPT 
has as part of its membership one representative from each 
of the working-level IPTs who act as a linking pin with his 
own working-level IPT. Even though these teams are focused 
on a particular functional area, they are still multi- 
disciplinary. The supportability IPT should not be a team 
solely of logisticians but should have representatives from 
the disciplines that will influence the supportability of 
the item. 

Program IPTs are formed at the program level to manage and 
execute programs . 
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APPENDIX C: FACTORS TO SUSTAIN SUPPORT 

AND REDUCE COSTS 



In accordance with ASN (RD&A) memorandum of 14 February 
1996, the following four factors must be considered by Navy 
PMAs and their IPTs, Program Executive Officers (PEOs) , 
Direct Reporting Program Managers (DRPMs) , NAVAIR Systems 
Commanders, and the Navy Secretariat staff in establishing 
supportability requirements: total cost of ownership, 
maintenance concept, standardization, and supportability. 

An accurate picture of the total cost of ownership and 
cost relationships is necessary for cost reductions. Total 
cost of ownership includes all costs associated with the 
research, development, procurement, operation, logistical 
support and disposal of an individual weapons system. It 
includes the total support infrastructure that plans, 
manages, and executes the weapons system over its full life. 
Currently, decisions focus on a specific cost element, 
budget line, or product line without considering the impact 
on the rest of the infrastructure. For example savings in 
depot maintenance may increase the number of systems 
required in the pipeline to maintain adequate resources at 
the operational level. Similarly, design changes may 
marginally improve performance but dramatically drive up 
support equipment costs. 

The maintenance concept expresses the strategy for 
maintaining the platform and system at a defined level of 
readiness in support of the operational scenario. It 
includes preventive maintenance, corrective maintenance, and 
overhaul. Maintenance concepts for the platform, systems, 
and support equipment must consider maintainability at all 
maintenance levels and must be consistent. 

Standardization is intended to ensure the minimal 
variety and optimal interchangeability of technical 
information, training, equipment parts, and components. 
Achieving standardization is often in direct opposition to 
the use of performance specif ications and commercial or 
nondevelopmental items. A balance between these two ends of 
the spectrum is obtained by using business and technical 
judgement in determining how to reduce the total cost of 
ownership . 
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Supportability requirements must fully consider life 
cycle costs including possible short life spans resulting 
from technology insertions or obsolescence. Requirements 
must also consider the risk of service period extensions. 
Planning must include the post production phase. PMAs must 
identify the most cost effective approach to supporting the 
system when fielded and assure that the required support 
elements, data, and information are developed and acquired. 
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APPENDIX D: SUPPORTABILITY STRATEGY 

STEPS 



"Contracting for Supportability" (NAVAIR, 1998) identifies 
five steps to be used to establish a supportability strategy 
for acquisition programs for new systems, major and minor 
modifications or upgrades, and commercial and 
nondevelopmental items. The following steps should be 
tailored for each type of acquisition program. 

1 . Develop Strategy and Initial Support Requirements 

The APML' s first action is to determine the acquisition 
logistics strategy consistent with the overall program 
acquisition strategy. Major considerations in determining 
the acquisition logistics strategy are the type of 
acquisition, system complexity, acquisition phase, 
availability of historic data, and time and resources 
available. The availability, accuracy, and relevance of 
experience and historical databases on similar existing 
systems are crucial for accomplishment of some tasks. 
Available databases must be examined to determine if 
extensive work is needed to provide focus or relevancy. The 
acquisition logistics strategy should be periodically 
reviewed and updated to reflect any changes to the program. 
After the initial requirements are selected, further 
refinement is needed to concentrate effort in high leverage 
areas. Specific models and associated databases may be 
considered and identified a.t this time. 

2 . Design Interface with Interrelated Efforts 

The APML must plan how to interface logistics 
requirements with the engineering community. Key related 
programs include reliability engineering, maintainability 
engineering, value engineering, human systems integration, 
system safety engineering, and transportability engineering. 
The acquisition logistics program is integrated with these 
related programs to prevent duplication of analyses and data 
and to ensure that analyses are performed in a timely 
manner. Logistics data is sometimes based on, and should be 
traceable to, systems engineering activities. Design and 
performance information can be captured, disseminated, and 
formally controlled to serve as an audit trail for logistics 
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support planning, trade-off analyses, and documentation 
preparation . 

3 . Select Logistics Products to be Developed and 
Delivered 

The APML must determine what acquisition logistics 
products are to be delivered and how they will be delivered 
(magnetic tape, disk, hard copy) . The importance of 
acquiring the appropriate data must be emphasized in keeping 
with the evolving policies regarding specifications and 
standards reform and with the thrust to reduce data 
requirements. The right data can be critical. Unnecessary 
data is simply wasteful. 

4 . Determine Supportability Costs 

After the APML has developed all tasks and data 
selection has been completed, he must determine the costs 
associated with the effort and document funding 
requirements . 

5 . Finalize Acquisition Logistics Strategy and 
Document in Acquisition Logistics Plan and 
Statement of Work 

These actions are not independent, and careful review 
is required to ensure consistency. After the acquisition 
logistics plan becomes part of the procurement request for 
the end item, the contractor responds with his support plan. 
This ensures acquisition logistics will be integrated with 
the total acquisition program. 
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APPENDIX E: NALDA 



NALDA has been operational from the early 1980s. It 
evolved from a need for improved data analysis capabilities 
to support Fleet aviation weapon systems management. NALDA 
today is the Navy and Marine Corps central aviation 
maintenance and logistics automated information system. It 
provides an on-line, integrated life cycle logistics 
readiness and operational weapons systems database and tools 
to sustain critical support analysis. NALDA is accessed and 
used daily by Navy/Marine aviation headquarters, fleet and 
field activities. This system provides accessible, 
comprehensive, accurate and timely aviation logistics data 
analysis and reporting capabilities to support fleet 
readiness, through sustainability of sophisticated and 
complex Naval Aviation weapons and associated support 
equipment and systems. NALDA applications encompass the 
logistics planning, management, administration, budgeting, 
and resource allocation in support of air weapon systems and 
related support equipment. The intent of NALDA is to 
support naval aviation logistics as established by the Naval 
Aviation Maintenance Program 7 (NAVAIR, 1997) . 

A. NALDA Phase I 

The NALDA design has followed a phased architecture . 
Phase I is currently operational. The NALDA system is 
composed of hierarchical Data Base Management System 2000 
(S2K) databases. The primary source of data is the AV3M 
data received via Naval Sea Logistics Command (NSLC) . 
Secondary sources come from NADEPs and ASO. It operates on 
the AMDAHL 5995 mainframe located at the Defense MegaCenter 
in Mechanicsburg PA. The telecommunications network 
presently consists of local dial-up and WATS lines. 



7 For additional information on the Naval Aviation 
Maintenance Program, see OPNAV 4790. 2G. This instruction 
provides detailed requirements and guidance for all facets 
of the three levels of aircraft maintenance. 
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B. NALDA Phase II 



The current NALDA Phase I architecture is characterized 
by several proprietary stovepipe systems. These systems 
often lack interfaces, and offer redundant and conflicting 
information. Additionally, many of the applications are 
non-Year 2000 compliant. Phase II will address these 
deficiencies . 

Phase I will be migrated in two increments to a 
client/server architecture on the SP2 machines located at 
Patuxent River and employing the ORACLE RDBMS. Increment A 
is in work with plans to bring it on line 30 June 1998. 

This discussion will focus on Increment A, as this is where 
the LMDSS capability is introduced. NALDA II users will 
establish a link to a NALDA Web Page via the Internet using 
commercial Web browsers and telecommunications software. 

NALDA II provides an Integrated Data Environment (IDE) 
which will include the functionality of the systems from 
Phase I with expanded capabilities and incorporated into new 
systems. The goal is to create and store data once and use 
it many times. Phase II will include the following: 1) a 
Logistics Support Analysis Record; 2) an accurate 
Configuration Management Information System/Joint Logistics 
Systems Center software for aviation weapons systems - 
configuration management is considered to be one of the 
fleet's priorities to improve readiness and safety of 
flight; 3) the LMDSS, the Navy's primary decision support 
system to achieve cost-effective logistics management, more 
timely (daily) receipt of fleet AV3M and configuration data, 
cost-effective consolidation of central, upline AV3M data 
systems, and the ability to access centralized fleet- 
wide, near real-time, operational/readiness data from NALDA; 

4) Aircraft Inventory and Readiness Reporting System 
(AIRRS) ; 5) Reliability Centered Maintenance (RCM) ; 6) 
Visibility and Management of Operating/Support Cost Programs 
(VAMOSC) ; 7) Technical Data including Joint Computer Aided 
Logistics (JCALS) Interface; 8) Airborne Weapons Information 
System (AWIS) ; 9) Metrology Automated System for Uniform 
Recall and Reporting (MEASURE); 10) Affordable Readiness 
Metrics/Total Cost-Decision Support System (TC-DSS) ; 11) 

Level of Repair Analysis (LORA); and 12) other interfaces 
and applications as identified in life cycle documentation. 
Ultimately, the IDE, essentially a logistics data warehouse, 
will contain product definition, ILS acquisition, in-service 
management, fleet and depot maintenance, analysis, supply, 
cost, configuration management status reporting, and other 
data. All current and future NALDA applications will be 
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written against the single IDE data structure (NAVAIR, 
1997 } . 
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APPENDIX F: QUESTIONNAIRE 



This questionnaire is separated into four parts. Part One was used 
as part of a telephone interview. Parts Two through Four were mailed to 
the respondents to be filled out and then returned. 



PHONE INTRODUCTION 

Thank you for your contribution to the research project of Aerospace 
Maintenance Duty Officers LCDR Carolynn Snyder and LCDR Ellen Moore. We 
are currently pursuing Master of Science Degrees in Management at the 
Naval Postgraduate School, Monterey. Carolynn is a student in 
Information Technology and Ellen is a student in Material Logistics 
Support. We are analyzing logistics decision support in Navy aviation 
system program management. The quality of our review depends on your 
input . 

Specifically, we are looking at the LMDSS system designed to facilitate 
continuous action by NAVAIR logistics management teams to measurably 
reduce the life cycle support costs of aviation systems while protecting 
readiness. As a NALDA Phase II application, it incorporates data from 
existing maintenance, flight, cost, and material data bases into a 
repeatable decision making process. LMDSS is designed to enable 
logistics managers to answer the following: 

1) How am I doing? (performance versus plan) 

2) What are my current and future support cost and readiness 
drivers? 

3) What can I do about it? 

4) How much will the solution cost? 

5) What is the payback period? 

We want to ensure LMDSS meets your needs. We intend to propose how the 
Navy can measure if LMDSS measurably reduces the life cycle support costs 
of aviation systems (LMDSS objective) . This questionnaire will help us 
answer the following: 

1) Does the LMDSS architecture have the capability of satisfying 
APMLs ? 

2) What information does an APML use to make decisions?, and 

3) Does LMDSS provide that information? 



Ill 



date: 



PART ONE (Phone response from LMDSS USERS AND NON-USERS) 
completed by interviewer 

A. Identification information: 

1. Name: 

2 . Phone : 

3. e-mail: 

4 . Address : 



5. Job Title: 

6. Brief description of job: 



7. Do you work with new aviation systems, sustaining existing systems, or 
both? (circle) 



B. How often do you use the following tools to perform your job? In the 
capacity of identifying requirements, analyzing requirements or both? 



N: never 

I: infrequently 

M: monthly 

W: weekly 

D: daily 

DK: don't know 



Id: identify requirements 
A: analyze requirements 
B: both 

DK: don't know 



8. Logistic Support Analysis 

9 . Raw data 

10. Model (s) 



(MIL-STD-1388) N I 

N I 
N I 



M 


W 


D / DK // 


Id 


A 


B/DK 


M 


W 


D / DK // 


Id 


A 


B/DK 


M 


W 


D / DK // 


Id 


A 


B/DK 



if so, which one(s) ? 



11. Checklist 

if so, how was it developed? 



N I M W D / DK // Id A B / DK 
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12. 


Intuition/experience 


N 1 


1 M 


W 


D/DK 


II 


Id 


A 


B/DK 


13. 


other: 


N 1 


1 M 


W 


D/DK 


II 


Id 


A 


B/DK 



14. Do you know what LMDSS, Logistics Management Decision Support 
System, is? (circle) 

Yes No 

15. Have you previously or do you currently use LMDSS? (circle) 

Yes No 

(skip next question if previous answer is No) 

16. If you use LMDSS, how often? Infrequently Monthly Weekly Daily Don’t Know 

17. a. How often do you interface with project engineer (s)? 

N I M W D / DK // Id A B / DK 

b. What for? 

c. How (e-mail, phone, conference, etc.): 

18. a. How often do you interface with project analyst (s)? 

N I M W D / DK // Id A B / DK 

b. What for? 

c. How (e-mail, phone, conference, etc.) 

19. a. How often do you interface with those who influence the program 
budget? 

N I M W D / DK // Id A B / DK 

b. Who? 

c. What for? 

d. How (e-mail, phone, conference, etc.) 

20. Other : 

a. Who? N I M W D/DK // Id AB/DK 

b. What for? 

c. How (e-mail, phone, conference, etc.) 

21. How do you measure life cycle costs? 
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This questionnaire has been developed for APMLs assigned to aviation 
system programs. Do you know of anyone else you feel would make a 
valuable contribution to our study, particularly anyone involved with 
logistics processes in aviation systems program management? 

Name : 

Phone : 

Address : 



Name : 
Phone : 
Address : 



Name : 
Phone : 
Address : 
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(MAIL) .... personalized address 

Thank you again for your contribution to the research project of 
Aerospace Maintenance Duty Officers LCDR Carolynn Snyder and LCDR Ellen 
Moore. As we discussed by phone (date) , we are currently pursuing Master 
of Science Degrees in Management at the Naval Postgraduate School, 
Monterey. Carolynn is a student in Information Technology and Ellen is a 
student in Material Logistics Support. We are analyzing logistics 
decision support in Navy aviation system program management. The quality 
of our review depends on your input. 

Specifically, we are looking at the LMDSS system designed to facilitate 
continuous action by NAVAIR logistics management teams to measurably 
reduce the life cycle support costs of aviation systems while protecting 
readiness. As a NALDA Phase II application, it incorporates data from 
existing maintenance, flight, cost, and material data bases into a 
repeatable decision making process. LMDSS is designed to enable 
logistics managers to answer the following: 

1) How am I doing? (performance versus plan) 

2) What are my current and future support cost and readiness 
drivers? 

3) What can I do about it? 

4) How much will the solution cost? 

5) What is the payback period? 

We want to ensure LMDSS meets your needs. We intend to propose how the 
Navy can measure if LMDSS measurably reduces the life cycle support costs 
of aviation systems (LMDSS objective). This questionnaire will help us 
answer the following: 

1) Does the LMDSS architecture have the capability of satisfying 
APMLs? 

2) What information does an APML use to make decisions?, and 

3) Does LMDSS provide that information? 



Please feel free to address any questions or comments to: 

LCDR Ellen Moore/phone: 408-657-0891/email: eemoore@nps.navy.mil, 
or LCDR Carolynn Snyder /phone : 408-393-9567/email: cmsnyder@nps.navy.mil. 

The questionnaire is divided into four parts: 

Part ONE: Information provided by phone (please review and provide 
additional comments as desired) . 

Part TWO: Written response from LMDSS users. 

Part THREE: Written response from those who have not used LMDSS. 

Part FOUR: Written response, job information needs (LMDSS users and 
non-users) . 
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Please mark each section with a legible pen or pencil. Attach additional 
pages as required. 



PART TWO (LMDSS Users) 

C. How frequently do you work with the following logistics concerns? 
Additionally indicate if it is in the capacity of identifying 
requirements, analyzing requirements or both, (circle the best answer) 

22. Logistic criteria as input to system design 
(reliability/maintainability goals /objectives ) 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

.£>. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 



23. Human factors concerns 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 

24. Failure mode, effects, and critical analysis 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON'T KNOW 

REQUIREMENTS REQUIREMENTS 

25. Failure reporting, analysis, and corrective-action system 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 

26. Provisioning needs/alternatives 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 
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27. Compatibility with existing system 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

28. Configuration Management 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

29. Training and training support 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 



30. Manpower and personnel 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

31. Supply support, spares 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

32. Inventory level analysis 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 



DON'T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 
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33. Transportation, packaging or storage 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 



34 . Test equipment 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

35. Support equipment 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

36. Computer resource support 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

37. Facilities, requirements 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

38. Facilities, location 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 



DON'T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 
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39. Data, reports requirements 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity : IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 

40. Maintenance planning (scheduled versus unscheduled plan) 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 



41. Level of repair analysis (0 versus I versus D-levels) 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 



42. Operating environment issues 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 

43. Cost-drivers 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 

44. Readiness degraders 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 
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45. Cycle time to repair components 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 

46. other: 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 



47. How often do you use LMDSS? 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 



**** jf y 0 u do not use LMDSS then the next portion of this questionnaire 
has been sent to you in error. STOP NOW and call LCDR Ellen Moore/phone 
408-657-0891 or LCDR Carolynn Snyder/phone 408-393-9567 for the correct 
questionnaire for NON-USERS. 

If you do use LMDSS, please continue with the questionnaire. 



D. How often do you use the following types of LMDSS queries? (circle 
the best answer) 



4 ^ 

00 


Summary data for end 


. items 










NEVER INFREQUENTLY 


MONTHLY 


WEEKLY 


DAILY 


DON’T KNOW 


49. 


Reliability summary 


parameters 








NEVER INFREQUENTLY 


MONTHLY 


WEEKLY 


DAILY 


DON’T KNOW 


50. 


Supportability summary parameters 








NEVER INFREQUENTLY 


MONTHLY 


WEEKLY 


DAILY 


DON’T KNOW 
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51. Cost summary parameters 



NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

52. Trend analysis (problems and causes) 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

53. Component and/or end-item cost data, specifically: Annual Operations 
and Support Costs 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

54. Component and/or end-item cost data, specifically: Labor Cost History 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

55. Component and/or end-item cost data, specifically: Item Value to 
Depot Repair Cost 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

56. Component and/or end-item cost data, specifically: Budget Analysis 
(OP-20 Report) 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

57. Component and/or end-item cost data, specifically: Inflation Factors 

NEVER INFREQUENTLY MONTHLY .WEEKLY DAILY DON’T KNOW 

58. Component and/or end-item cost data, specifically: Item Value to 
Labor Cost 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

59. Candidate Identification Function, specifically: Detailed Component 
Report 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

60. Candidate Identification Function, specifically: Wholesale System 
Demand 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
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61. Candidate Identification Function, specifically: 
Trends 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

62. Candidate Identification Function, specifically: 

. NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

63. Candidate Identification Function, specifically: 
Investment 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

64. Candidate Identification Function, specifically: 
Wait Reports 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

65. Candidate Identification Function, specifically: 
Reports 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

66. Candidate Identification Function, specifically : 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

67. Candidate Identification Function, specifically: 
Between Failure Report 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

68. Engine Repair Cost 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

69. Flight Hours Since Last Engine Repair 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

70. Engine Demand Forecasting 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 



Material Issue 

DON'T KNOW 

Supply Synopsis 

DON'T KNOW 

Wholesale System 

DON'T KNOW 

Average Customer 

DON’T KNOW 

Backorder History 

DON’T KNOW 

NAVICP NSN Snapshot 

DON’T KNOW 

Mean Flight Hour 

DON'T KNOW 



DON’T KNOW 



DON'T KNOW 



DON'T KNOW 
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71. Engine Overview 



NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 



72. 



73. 



74. 



75. 



76. 



77 . 



78. 



79. 



80. 



81. 



Reference Information: Aircraft Inventory and Fatigue Life 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

Reference Information: Code Definition 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

Reference Information: Aircraft Engine Installation and Ownership 



NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

Reference Information: Production Load and Run Statistics 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

Reference Information: Possible Courses of Action 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

Reference Information: Organization Codes /Job Count 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 



Reference Information: NIIN/CAGE/Part Number Cross Reference 



NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

Reference Information: TEC Information 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

Reference Information: SALTS File Information 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

Reference Information: Data Dictionary 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 
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E. Please describe your experience with the following LMDSS 
(circle the best answer) 

Please include comments to clarify answers. 

82. Interface 

STRONGLY DISLIKE NEUTRAL LIKE STRONGLY 
DISLIKE LIKE 

comments : 



83. Help function 

STRONGLY DISLIKE NEUTRAL LIKE STRONGLY 

DISLIKE LIKE 

comments: 

84. Analysis Tools 

STRONGLY DISLIKE NEUTRAL LIKE STRONGLY 

DISLIKE LIKE 

comments: 



85. Report Presentation/format 

STRONGLY DISLIKE NEUTRAL LIKE STRONGLY 
DISLIKE LIKE 



comments: 



86. Time required getting what is needed 



STRONGLY 


DISLIKE 


NEUTRAL 


LIKE 


STRONGLY 


DISLIKE 








LIKE 



comments: 



functions? 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 
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87. Ease of getting what is needed 



STRONGLY 


DISLIKE 


NEUTRAL 


LIKE 


STRONGLY 


DISLIKE 








LIKE 



DON’T KNOW 



comments : 



88. Training 

STRONGLY DISLIKE NEUTRAL LIKE STRONGLY DON’T KNOW 

DISLIKE LIKE 

comments: 



89. 'Accessibility (when desired) 

STRONGLY DISLIKE NEUTRAL LIKE STRONGLY DON’T KNOW 

DISLIKE LIKE 



comments : 



90. Accessibility (server access) 



STRONGLY DISLIKE NEUTRAL LIKE STRONGLY 
DISLIKE LIKE 



DON’T KNOW 



comments : 



91. Accessibility (password access) 

STRONGLY DISLIKE NEUTRAL LIKE STRONGLY DON’T KNOW 

DISLIKE LIKE 

comments: 



92. Provides the information I need 

STRONGLY DISLIKE NEUTRAL LIKE STRONGLY DON’T KNOW 

DISLIKE LIKE 

comments: 
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F. Please describe your experience using the LMDSS data 
available, (circle the best answer) 

Please include comments to clarify answers. 



93. Data meets my needs 



STRONGLY DISAGREE 
DISAGREE 



NEUTRAL AGREE STRONGLY 

AGREE 



comments : 



94 . Data is accessible 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 



comments : 



95. Data is accurate/consistent 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 

comments : 



96. Data is detailed enough 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 



comments: 



97. The exact data meaning is clear 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 



comments: 



currently 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 
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G. Please describe your experience using other data currently available, 
(circle the best answer) 

Please include comments to clarify answers. 



98. Data meets my needs 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 

DISAGREE AGREE 

comments : 

99. Data is accessible 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 

DISAGREE AGREE 

comments : 

100. Data is accurate/consistent 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 

DISAGREE AGREE 

comments: 

101. Data is detailed enough 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 

DISAGREE AGREE 

comments: 



102. The exact data meaning is clear 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 



comments: 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



Please continue now with PART FOUR, Section J, Job Information Needs 



Please mark each section with a legible pen or pencil. Attach additional 
pages as required. 



PART THREE (LMDSS Non-users) 

H. How frequently do you work with the following logistics concerns? 
Additionally indicate if it is in the capacity of identifying 
requirements, analyzing requirements or both, (circle the best answer) 



103. Logistic criteria as input to system design 
(reliability /maintainability goals /objectives) 



.a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON'T KNOW 

REQUIREMENTS REQUIREMENTS 

104. Human factors concerns 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON'T KNOW 

REQUIREMENTS REQUIREMENTS 

105. Failure mode, effects, and critical analysis 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 



106. Failure reporting, analysis, and corrective-action system 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 

107. Provisioning needs/alternatives 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 
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108. Compatibility with existing system 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

109. Configuration Management 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

110. Training and training support 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

111. Manpower and personnel 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

112. Supply support, spares 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

113. Inventory level analysis 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 



DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 
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114. Transportation, packaging or storage 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 



115. Test equipment 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

116. Support equipment 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

117. Computer resource support 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

118. Facilities, requirements 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 

119. Facilities, location 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 

b. capacity: IDENTIFY ANALYZE BOTH 

REQUIREMENTS REQUIREMENTS 



DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON'T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 

DON’T KNOW 
DON’T KNOW 
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120. Data, reports requirements 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON'T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON'T KNOW 

REQUIREMENTS REQUIREMENTS 

121. Maintenance planning (scheduled versus unscheduled plan) 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS ' REQUIREMENTS 

122. Level of repair analysis (0 versus I versus D-levels) 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 



123. Operating environment issues 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 

124. Cost-drivers 



a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 

125. Readiness degraders 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 

b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 

REQUIREMENTS REQUIREMENTS 
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126. Cycle time to repair components 



a 


. frequency: 


NEVER INFREQUENTLY 


MONTHLY 


WEEKLY DAILY 


DON’T KNOW 


b 


. capacity: 


IDENTIFY 


ANALYZE 


BOTH 


DON’T KNOW 




REQUIREMENTS 


REQUIREMENTS 




other: 










a 


. frequency: 


NEVER INFREQUENTLY 


MONTHLY 


WEEKLY DAILY 


DON’T KNOW 


b 


. capacity: 


IDENTIFY 


ANALYZE 


BOTH 


DON’T KNOW 



REQUIREMENTS REQUIREMENTS 



128. How often do you use LMDSS? 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 



**** If you use LMDSS then the next portion of this questionnaire has 
been sent to you in error. STOP NOW and call LCDR Ellen Moore/phone 408 
657-0891 or LCDR Carolynn Snyder/phone 408-393-9567 for the correct 
questionnaire for NON-USERS. 

If you do not use LMDSS, please continue with the questionnaire. 



129. Why do you not use LMDSS? (check mark all that apply) 

I didn't know it existed 

My PC won't support LMDSS 

I've received no training 

I don't need the information LMDSS provides 

It takes too long to get what I need 

It's too difficult to get what I need 

It doesn't provide me the information I need 

What information do you need that isn't provided? 
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I. Please describe your experience using other data currently available, 
(circle the best answer) 

Please include comments to clarify answers. 



130. Data meets my needs 

STRONGLY DISAGREE NEUTRAL 
DISAGREE 

comments: 



AGREE 



STRONGLY 

AGREE 



131. Data is accessible 

STRONGLY DISAGREE 
DISAGREE 



NEUTRAL 



AGREE 



STRONGLY 

AGREE 



comments: 



132. Data is accurate/consistent 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 

comments: 



133. Data is detailed enough 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 

comments: 



134. The exact data meaning is clear 

STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 

comments: 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



Please continue now with PART FOUR, Section J, Job Information Needs 



Please mark each section with a legible pen or pencil. Attach additional 
pages as required. 



PART FOUR (Job Information Needs) 

J. Indicate how useful the following information elements are (or would 
be) in performing your job. If an element is not provided, please 
include as an addition, (circle the best answer) 



135. Summary data; End Item 

NEUTRAL 



NOT AT ALL 
USEFUL 



NOT 

USEFUL 



136. Summary data; Claimant 

NEUTRAL 



NOT AT ALL 
USEFUL 



NOT 

USEFUL 



137. Summary data; Organization 

NEUTRAL 



NOT AT ALL 
USEFUL 



NOT 

USEFUL 



138. Summary data; BCM Report 

NEUTRAL 



NOT AT ALL 
USEFUL 



NOT 

USEFUL 



SLIGHTLY 

USEFUL 



SLIGHTLY 

USEFUL 



SLIGHTLY 

USEFUL 



SLIGHTLY 

USEFUL 



139. Reliability summary parameters 



NOT AT ALL 
USEFUL 



NOT 

USEFUL 



NEUTRAL 



SLIGHTLY 

USEFUL 



140. Supportability summary parameters 



NOT AT ALL 
USEFUL 



NOT 

USEFUL 



NEUTRAL 



141. Cost summary parameters 

NEUTRAL 



NOT AT ALL 
USEFUL 



NOT 

USEFUL 



142. Emerging Problems 



NOT AT ALL 
USEFUL 



NOT 

USEFUL 



NEUTRAL 



SLIGHTLY 

USEFUL 



SLIGHTLY 

USEFUL 



SLIGHTLY 

USEFUL 



EXTREMELY 

USEFUL 



EXTREMELY 

USEFUL 



EXTREMELY 

USEFUL 



EXTREMELY 

USEFUL 



EXTREMELY 

USEFUL 



EXTREMELY 

USEFUL 



EXTREMELY 

USEFUL 



EXTREMELY 

USEFUL 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 



DON’T KNOW 
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143. Common Equipment 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 



144. Trend analysis (problems and causes) 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

145. Component and/or end-item cost data, specifically: Annual Operations 
and Support Costs 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

146. Component and/or end-item cost data, specifically: Labor Cost 
History 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

147. Component and/or end-item cost data, specifically: Item Value to 
Depot Repair Cost 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

148. Component and/or end-item cost data, specifically: Budget Analysis 
(OP-20 Report) 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 



149. Component and/or end-item cost data, 



specifically: Inflation Factors 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

150. Component and/or end-item cost data, specifically: Item Value to 
Labor Cost 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

151. Candidate Identification Function, specifically: Detailed Component 
Report 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 
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152. Candidate Identification Function, specifically: 
Demand 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 

153. Candidate Identification Function, specifically: 
Trends 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 

154. Candidate Identification Function, specifically: 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 

155. Candidate Identification Function, specifically: 
Investment 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 

156. Candidate Identification Function, specifically: 
Wait Reports 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 

157. Candidate Identification Function, specifically: 
Reports 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 

158. Candidate Identification Function, specifically: 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 

159. Candidate Identification Function, specifically: 
Between Failure Report 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 



Wholesale System 
DON’T KNOW 
Material Issue 
DON’T KNOW 
Supply Synopsis 
DON’T KNOW 
Wholesale System 
DON’T KNOW 
Average Customer 
DON’T KNOW 
Backorder History 
DON’T KNOW 

NAVICP.NSN Snapshot 

DON’T KNOW 

Mean Flight Hour 

DON’T KNOW 
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specifically: Wait Time 



160. Candidate Identification Function, 

Maintenance Impact 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 



161. Candidate Identification Function, specifically: Average Days to 
Receipt 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 



162. Candidate Identification Function, specifically: Planned versus 
Actual Opportunity Costs 



NOT AT ALL 
USEFUL 


NOT 

USEFUL 


NEUTRAL 


SLIGHTLY 

USEFUL 


EXTREMELY 

USEFUL 


DON’T KNOW 


163. Engine 


Repair 


Cost 








NOT AT ALL 
USEFUL 


NOT 

USEFUL 


NEUTRAL 


SLIGHTLY 

USEFUL 


EXTREMELY 

USEFUL 


DON’T KNOW 


164. Flight 


Hours Since Last 


Engine Repair 




NOT AT ALL 
USEFUL 


NOT 

USEFUL 


NEUTRAL 


SLIGHTLY 

USEFUL 


EXTREMELY 

USEFUL 


DON’T KNOW 


165. Engine 


Demand 


Forecasting 






NOT AT ALL 
USEFUL 


NOT 

USEFUL 


NEUTRAL 


SLIGHTLY 

USEFUL 


EXTREMELY 

USEFUL 


DON’T KNOW 


166. Engine 


Overview 








NOT AT ALL 
USEFUL 


NOT 

USEFUL 


NEUTRAL 


SLIGHTLY 

USEFUL 


EXTREMELY 

USEFUL 


DON’T KNOW 


167. Engine 


Removal 


Trend 








NOT AT ALL 
USEFUL 


NOT 

USEFUL 


NEUTRAL 


SLIGHTLY 

USEFUL 


EXTREMELY 

USEFUL 


DON’T KNOW 


168. Flight 


Hour Since Engine 


Repair at 


Removal 




NOT AT ALL 
USEFUL 


NOT 

USEFUL 


NEUTRAL 


SLIGHTLY 

USEFUL 


EXTREMELY 

USEFUL 


DON’T KNOW 
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169. Reference Information: Aircraft Inventory and Fatigue Life 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

170. Reference Information: Code Definition 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON'T KNOW 

USEFUL USEFUL USEFUL USEFUL 

171. Reference Information: Aircraft Engine Installation and Ownership 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

172. Reference Information: Production Load' and Run Statistics 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

173. Reference Information: Possible Courses of Action 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

174. Reference Information: Organization Codes /Job Count 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

175. Reference Information: NIIN/CAGE/Part Number Cross Reference 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

176. Reference Information: TEC Information 



NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY • DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 



177. Reference Information: SALTS File Information 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 

178. Reference Information: Data Dictionary 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 

USEFUL USEFUL USEFUL USEFUL 
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179. Other: 



NOT AT ALL 


NOT 


NEUTRAL 


SLIGHTLY 


EXTREMELY 


DON'T KNOW 


USEFUL 


USEFUL 




USEFUL 


USEFUL 





We value your input, thank you again for your contribution to our 
research . 

180. May we contact you to clarify or expand the information provided? 

YES NO 



Please return the questionnaire by mail to the following by 27 March 
1998. (in self addressed stamped envelope provided) 

LCDR Ellen Moore 
SMC 1689, Herman Hall 
Naval Postgraduate School 
Monterey, CA 93943 

Please feel free to address any questions or comments to: 

LCDR Ellen Moore/phone: 408-657-0891/email: eemoore@nps.navy.mil 

or LCDR Carolynn Snyder/phone: 408-393-9567/email : cmsnyder@nps.navy.mil. 



Please include any additional comments, concerns, or questions below or 
on attached pages. 
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APPENDIX G: QUESTIONNAIRE RESULTS 



PART ONE (Phone response from LMDSS Users and Non-users) 

A Identifying Information: 

1 Name 

2 Phone 

3 email 

4 Address 

5 Job Title 

6 Brief Description of job 

7 Do you work with new aviation systems, sustaining existing systems or both? 

new existing both 

1 2 5 

B How often do you work with the following tools to perform your job? 

In the capacity of identifying the requirements, analyzing the requirements, or both? 

8 Logistics Support Analysis (MlL-STD-1388) 



never 


infrequently 


monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


3 


4 


1 


0 


0 


0 


1 


1 


2 


1 


9 Raw Data 


never 


infrequently 


monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 


0 


0 


1 


7 


0 


0 


1 


6 


1 


10 Models 


never 


infrequently 


monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


4 


0 


0 


0 


2 


1 


0 


0 


3 


1 


11 Checklists 


never 


infrequently 


monthly 


weekly 


daily 


don’t know 


identify 


analyze 


both 


don't know 


3 0 

12 Intuition/Experience 


0 


0 


4 


0 


0 


0 


4 


0 


never 


infrequently 


monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 


0 


0 


0 


8 


0 


0 


1 


7 


0 


13 Other 


never 


infrequently 


monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 0 0 
14 Do you know what the LMDSS is? 


0 


4 


2 


0 


1 


3 


2 



yes no 

8 0 

15 Have you previously or do you currently use the LMDSS? 

yes no 

1 7 

16 If you use the LMDSS, how often? 

infrequently monthly weekly daily don't know 
0 0 0 1 0 

17 How often do you interface with project engineer(s)? 



never infrequently monthly weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


10 0 0 


7 


0 


0 


1 


7 


0 


18 How often do you interface with project analyst(s)? 


never infrequently monthly weekly 


daily 


don’t know 


identify 


analyze 


both 


dont know 


0 0 0 0 7 0 

1 9 How often do you interface with those who influence the program budget? 


0 


1 


6 


0 


never infrequently monthly weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 10 3 


4 


0 


0 


0 


4 


3 


20 How often do you interface with others? 


never infrequently monthly weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 0 0 2 
21 How do you measure life cycle costs? 


6 


0 


0 


0 


6 


0 
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PART TWO (LMDSS Users) 

C How frequently do you work with the following logistics concerns? 

Additionally, indicate if it is in the capacity of identifying requirements, analyzing requirements, or both. 
22 Logistics criteria as input to system design 



never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


23 Human factors concerns 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 1 0 


0 


0 


0 


0 


0 


1 


0 


24 Failure mode, effects, and ctirical analysis 

never infrequently monthly weekly 


daily 


don't know 


identify 


analyze 


both 


dont know 


0 1 0 


0 


0 


0 


. 0 


0 


1 


0 


25 Failure reporting, analysis, and corrective action system 
never infrequently monthly weekly daily 


don't know 


identify 


analyze 


both 


dont know 


0 1 0 


0 


0 


0 


0 


0 


1 


0 


26 Provisioning needs/altematives 


never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


27 Compatiblity with existing systems 


never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


28 Configuration Management 


never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


dont know 


0 1 0 


0 


0 


0 


0 


0 


1 


0 


29 Training and training support 

never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


30 Manpower and personnel 


never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


31 Supply support/spares 


never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


32 Inventory level analysis 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


33 Transportation, packaging or storage 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 1 0 


0 


0 


0 


. 0 


0 


1 


0 


34 Test equipment 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


35 Support equipment 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


36 Computer Resource support 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 1 0 


0 


0 


0 


0 


0 


1 


0 


37 Facilities, requirements 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 1 0 


0 


0 


0 


0 


0 


1 


0 


38 Facilities, location 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 1 0 


0 


0 


0 


0 


0 


1 


0 


39 Data, reports requirements 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 
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40 Maintenance planning (scheduled versus unscheduled plan) 



never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


41 Level of repair analysis (O versus I versus D-Ievels) 












never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 1 0 


0 


0 


0 


0 


0 


1 


0 


42 Operating environment issues 
















never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 1 0 


0 


0 


0 


0 


0 


1 


0 


43 Cost-drivers 
















never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


44 Readiness degraders 
















never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don’t know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


45 Cycle time to repair components 
















never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 0 0 


0 


1 


0 


0 


0 


1 


0 


46 Other 
















never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 0 0 


0 


0 


0 


0 


0 


0 


0 


47 How often do you use LMDSS? 
















never infrequently monthly 


weekly 


daily 


don't know 










0 0 0 


0 


1 


0 










48 Summary data for end items 
















never infrequently monthly 


weekly 


daily 


don't know 










0 0 0 


0 


1 


0 










49 Reliability summary parameters 
















never infrequently monthly 


weekly 


daily 


don't know 










0 0 0 


0 


1 


0 










50 Suppo rtability summary parameters 
















never infrequently monthly 


weekly 


daily 


don't know 










0 0 0 


0 


1 


0 










51 Cost summary parameters 
















never infrequently monthly 


weekly 


daily 


don't know 










0 0 0 


0 


1 


0 










52 Trend analysis (problems and causes) 














never infrequently monthly 


weekly 


daily 


don't know 










0 0 0 


0 


1- 


0 











53 Component and/or end-item cost data, specifically: Annual Operating and Support Costs 

never infrequently monthly weekly. daily don't know 

0 0 0 0 1 0 

54 Component and/or end-item cost data, specifically: Labor Cost History 

never infrequently monthly weekly daily don't know 

0 0 0 0 1 0 

55 Component and/or end-item cost data, specifically: Item Value to Depot Repair Cost 

never infrequently monthly weekly daily don't know 

0 0 0 0 1 0 

56 Component and/or end-item cost data, specifically: Budget Analysis (OP-20 Report) 

never infrequently monthly weekly daily don't know 

0 1 0 0 0 0 

57 Component and/or end-item cost data, specifically: Inflation Factors 

never infrequently monthly weekly daily don't know 

0 1 0 0 0 0 

58 Component and/or end-item cost data, specifically: Item Value to Labor Cost 

never infrequently monthly weekly daily don't know 

0 1 0 0 0 0 
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59 Candidate Identification Function, specifically: Detail Component Report 

never infrequently monthly weekly daily don't know 

0 0 0 0 1 0 

60 Candidate Identification Function, specifically: Wholesale System Demand 

never * infrequently monthly weekly daily don't know 

0 1 0 0 0 0 

61 Candidate Identification Function, specifically: Material Issue Trends 

never infrequently monthly weekly daily don't know 

0 0 0 0 1 0 

62 Candidate Identification Function, specifically: Supply Synopsis 

never infrequently monthly weekly daily don't know 

0 1 0 0 0 0 

63 Candidate Identification Function, specifically: Wholesale System Investment 

never infrequently monthly weekly daily don't know 

0 1 0 0 0 0 

64 Candidate Identification Function, specifically: Average Customer Wait Reports 

never infrequently monthly weekly daily don't know 

0 1 0 0 0 0 

65 Candidate Identification Function, specifically: Backorder History Reports 

never infrequently monthly weekly daily don't know 

0 1 0 0 0 0 

66 Candidate Identification Function, specifically: NIIN NSN Snapshots 

never infrequently monthly weekly daily don't know 

0 0 0 0 1 0 

67 Candidate Identification Function, specifically: Mean Flight Hour Between Failure Report 



never infrequently 


monthly 


weekly 


daily 


don't know 


0 0 


0 


0 


1 


0 


68 Engine Repair Cost 










never infrequently 


monthly 


weekly 


daily 


don't know 


0 1 


0 


0 


0 


0 


69 Flight Hours Since Last Engine Repair 






never infrequently 


monthly 


weekly 


daily 


don’t know 


0 1 


0 


0 


0 


0 


70 Engine Demand Forecasting 








never infrequently 


monthly 


weekly 


daily 


don’t know 


0 1 


0 


0 


0 


0 


71 Engine Overview 










never infrequently 


monthly 


weekly 


daily 


don't know 


0 1 


0 


0 


0 


0 


72 Reference Information: Aircraft Inventory and Fatigue Life 




never infrequently 


monthly 


weekly 


daily 


don't know 


0 1 


0 


0 


0 


0 


73 Reference Information: Code Definition 






never infrequently 


monthly 


weekly 


daily 


don't know 


0 1 


0 


0 


0 


0 


74 Reference Information: Aircraft Engine Installation and Ownership 


never infrequently 


monthly 


weekly 


daily 


don't know 


0 1 


0 


0 


0 


0 


75 Reference Information: Production Load and Run Statistics 




never infrequently 


monthly 


weekly 


daily 


don't know 


0 1 


0 


0 


0 


0 


76 Reference Information: Possible Courses of Action 






never infrequently 


monthly 


weekly 


daily 


don't know 


0 1 


0 


0 


0 


0 


77 Reference Information: Organization Codes/Job Count 




never infrequently 


monthly 


weekly 


daily 


don't know 


0 0 


0 


0 


1 


0 
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78 Reference Information: NllN/CAGE/Part Number Cross Reference 



never 


infrequently 


monthly 


weekly 


daily 


don't know 


0 


0 


0 


^ 0 


1 


0 


79 Reference Information: TEC Information 






never 


infrequently 


monthly 


weekly 


daily 


don't know 


0 


0 


0 


0 


1 


0 


80 Reference Information: SALTS File Information 






never 


infrequently 


monthly 


weekly 


daily 


don't know 


0 


1 


0 


0 


0 


0 


81 Reference Information: Data Dictionary 






never 


infrequently 


monthly 


weekly 


daily 


don't know 


0 


1 


0 


0 


0 


0 


E Please describe your experience with the following LMDSS functions. 


82 Interface 












strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


0 


0 


1 


0 


0 


83 Help Function 










strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


0 


0 


1 


0 


0 


84 Analysis Tools 










strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


0 


0 


1 


0 


0 


85 Report presentation/format 








strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


0 


0 


1 


0 


0 


86 Time required getting what is needed 








strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


1 


0 


0 


0 


0 


87 Ease of getting what is needed 








strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


1 


0 


0 


0 


0 


88 Training 












strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




1 


0 


0 


0 


0 


0 


89 Accessibility (when desired) 








strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


0 


1 


0 


0 


0 


90 Accessibility (server access) 








strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


0 


1 


0 


0 


0 


91 Accessibility (password access) 


- 






strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


0 


0 


1 


0 


0 


92 Provides the information 1 


1 need 








strongly 


dislike 


neutral 


like 


strongly 


don't know 


dislike 








like 




0 


0 


0 


1 


0 


0 
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F Please describe your experience using the LMDSS data currently available. 
93 Data meets my needs 



strongly 


disagree 


neutral 


agree 


strongly 


don't know 


disagree 








agree 




0 


0 


0 


1 


0 


0 


94 Data is accessible 










strongly 


disagree 


neutral 


agree 


strongly 


don't know 


disagree 








agree 




0 


0 


0 


1 


0 


0 


95 Data is accurate/consistent 








strongly 


disagree 


neutral 


agree 


strongly 


don't know 


disagree 








agree 




1 


0 


0 


0 


0 


0 


96 Data is detailed enough 










strongly 


disagree 


neutral 


agree 


strongly 


don't know 


disagree 








agree 




0 


0 


0 


1 


0 


0 


97 The exact data meaning 


is clear 








strongly 


disagree 


neutral 


agree 


strongly 


don't know 


disagree 








agree 




0 


1 


0 


0 


0 


0 


G Please describe your experience using other data currently available. 


98 Data meets my needs 










strongly 


disagree 


neutral 


agree 


strongly 


dont know 


disagree 








agree 




0 


0 


0 


0 


1 


0 


99 Data is accessible 










strongly 


disagree 


neutral 


agree 


strongly 


don't know 


disagree 








agree 




0 


0 


0 


0 


1 


0 


1 00 Data is accurate/consistent 








strongly 


disagree 


neutral 


agree 


strongly 


don’t know 


disagree 








agree 




0 


0 


0 


0 


1 


0 


1 01 Data is detailed enough 










strongly 


disagree 


neutral 


agree 


strongly 


don't know 


disagree 








agree 




0 


0 


0 


0 


1 


0 


102 The exact data meaning is clear 








strongly 


disagree 


neutral 


agree 


strongly 


don't know 


disagree 








agree 




0 


0 


0 


0 


1 


0 



PART THREE (LMDSS Non-users) 

H How frequently do you work with the following logistics concerns? 



Additionally, indicate if it is in the capacity of identifying requirements, analyzing requirements, or both. 
103 Logistics criteria as input to system design 



never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 3 2 


0 


2 


0 


2 


1 


4 


0 . 


104 Human factors concerns 

never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 4 1 


0 


2 


0 


1 


3 


3 


0 


105 Failure mode, effects, and ctirical analysis 

never infrequently monthly weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 4 0 


1 


2 


0 


1 


2 


4 


0 


106 Failure reporting, analysis, and corrective action 
never infrequently monthly weekly 


system 

daily 


dont know 


identify 


analyze 


both 


dont know 


0 1 2 


2 


1 


1 


1 


1 


4 


1 
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1 07 Provisioning needs/altematives 



never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 1 2 


2 


1 


1 


1 


1 


4 


1 


108 Compatibly with existing systems 


never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 2 1 


1 


2 


0 


0 


2 


3 


1 


109 Configuration Management 


never infrequently monthly 


weekly 


daily 


don't know 


identify 


analyze 


both 


don't know 


0 2 0 


2 


3 


0 


0 


2 


5 


0 


1 1 0 Training and training support 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


don't know 


0 1 3 


2 


1 


0 


1 


1 


5 


0 


1 1 1 Manpower'and personnel 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


don't know 


0 3 4 


0 


0 


0 


2 


3 


2 


0 


112 Supply support/spares 

never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


don't know 


0 1 0 


3 


3 


0 


0 


1 


6 


0 


113 Inventory level analysis 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


don't know 


0 2 2 


2 


1 


0 


0 


1 


5 


1 


114 Transportation, packaging or storage 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


don't know 


0 4 0 


2 


1 


0 


1 


1 


4 


1 


1 1 5 Test equipment 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


don't know 


0 1 3 


2 


1 


' 0 


1 


1 


5 


0 


116 Support equipment 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


don't know 


0 1 2 


2 


2 


0 


1 


1 


5 


0 


117 Computer Resource support 

never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


don't know 


0 2 2 


2 


1 


0 


2 


1 


3 


1 


118 Facilities, requirements 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 4 2 


1 


0 


0 


0 


2 


5 


0 


119 Facilities, location 

never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 5 2 


0 


0 ' 


0 


2 


2 


2 


1 


120 Data, reports requirements 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 0 1 


2 


4 


0 


0 


1 


6 


0 


1 21 Maintenance planning (scheduled versus unscheduled plan) 

never infrequently monthly weekly daily dont know 


identify 


analyze 


both 


dont know 


1 2 0 


2 


2 


0 


0 


2 


4 


0 


1 22 Level of repair analysis (O versus I versus D-Ievels) 

never infrequently monthly weekly daily 


dont know 


identify 


analyze 


both 


dont know 


0 3 0 


2 


2 


0 


0 


1 


6 


0 


1 23 Operating environment issues 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 2 2 


2 


1 


0 


0 


2 


5 


0 


124 Cost-drivers 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 2 3 


0 


2 


0 


0 


2 


5 


0 


1 25 Readiness degraders 


never infrequently monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 2 1 


1 


3 


0 


1 


1 


5 


0 
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126 Cycle time to repair components 



never 


infrequently 


monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 


3 


2 


1 


1 


0 


0 


1 


5 


1 


127 Other 




















never 


infrequently 


monthly 


weekly 


daily 


dont know 


identify 


analyze 


both 


dont know 


0 


0 


1 


0 


1 


0 


0 


0 


2 


0 


128 How often do you use LMDSS? 
















never 


infrequently 


monthly 


weekly 


daily 


dont know 










7 


0 


0 


0 


0 


0 











129 Why do you not use LMDSS? (check all that apply) 



2 I didn't know it existed 

1 My PC wont support LMDSS 

4 IVe received no training 

0 I don't need the information LMDSS provides 

0 It takes too long to get what I need 

0 It’s too difficult to get what I need 

2 It doesnt provide me the information I need 

1 other 

I Please describe your experience using other data currently available. 



Data meets my needs 


strongly disagree 


neutral 


agree 


strongly 


dont know 


disagree 

0 0 


3 


3 


agree 

0 


0 


Data is accessible 


strongly disagree 


neutral 


agree 


strongly 


dont know 


disagree 

0 0 


2 


3 


agree 

1 


0 



132 Data is accurate/consistent 

strongly disagree neutral 

disagree 

0 2 2 



agree strongly don't know 
agree 

1 0 1 



1 33 Data is detailed enough 

strongly disagree neutral 

disagree 

0 2 2 



agree strongly don't know 
agree 

2 0 0 



134 The exact data meaning is clear 

strongly disagree neutral agree 

disagree . 

0 2 3 1 



strongly dont know 
agree 

0 0 



PART FOUR (Job Information Needs, LMDSS Users and Non-users) 

J Indicate how useful the following information elements are (or would be) in performing your job. 



135 Summary 


Data; End Item 










not at all 


not 


neutral 


slightly 


extremely 


dont know 


useful 


useful 




useful 


useful 




0 


0 


0 


2 


4 


1 


136 Summary 


Data; Claimant 










not at all 


not 


neutral 


slightly 


extremely 


dont know 


useful 


useful 




useful 


useful 




0 


0 


2 


1 


2 


2 


137 Summary 


Data; Organization 


■ 






not at all 


not 


neutral 


slightly 


extremely 


dont know 


useful ' 


useful 




useful 


useful 




0 


1 


0 


1 


5 


0 


138 Summary 


Data; BCM Report 








not at all 


not 


neutral 


slightly 


extremely 


dont know 


useful 


useful 




useful 


useful 




0 


1 


0 


2 


4 


0 
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139 



140 



141 



142 



143 



144 



145 



146 



147 



148 



149 



150 



151 



152 



Reliability summary parameters 




not at all 


not 


neutral 


slightly 


useful 


useful 




useful 


0 


0 


0 


1 


Supportability summary parameters 




not at all 


not 


neutral 


slightly 


useful 


useful 




useful 


0 


0 


0 


1 


Cost summary parameters 






not at all 


not 


neutral 


slightly 


useful 


useful 




useful 


0 


0 


0 


1 


Emerging Problems 






not at all 


not 


neutral 


slightly 


useful 


useful 




useful 


0 


0 


0 


0 


Common Equipment 






not at all 


not 


neutral 


slightly 


useful 


useful 




useful 


0 


0 


0 


1 


Trend Analysis (problems versus causes) 


not at all 


not 


neutral 


slightly 


useful 


useful 




useful 


0 


0 


0 


1 



extremely don't know 
useful 

6 0 

extremely don't know 
useful 

5 1 

extremely dont know 
useful 

6 0 

extremely dont know 
useful 

7 0 

extremely dont know 
useful 

4 2 

extremely dont know 
useful 

5 1 



Component and/or end-item cost data, specifically: Annual Operating and Support Costs 
not at all not neutral slightly extremely dont know 

useful useful useful useful 

0 0 0 0 7 0 



Component and/or end-item cost data, specifically: Labor Cost History 
not at all not neutral slightly extremely dont know 

useful useful useful useful 



0 0 0 



0 7 0 



Component and/or end-item cost data, specifically: Item Value to Depot Repair Cost 
not at all not neutral slightly extremely dont know 

useful useful useful useful 

0 0 0 0 5 2 



Component and/or end-item cost data, specifically: Budget Analysis (OP-20 Report) 
not at all not neutral slightly extremely dont know 

useful useful useful useful 

0 0 1 2 3 1 

Component and/or end-item cost data, specifically: Inflation Factors 
not at all not neutral slightly extremely dont know 

useful useful useful useful 

0 0 1 3 2 1 

Component and/or end-item cost data, specifically: Item Value to Labor Cost 
not at all not neutral slightly extremely dont know 

useful useful useful useful 

0 0 0 2 3 2 

Candidate Identification Function, specifically: Detail Component Report 
not at all not neutral slightly extremely dont know 

useful useful useful useful 

0 0 2 0 4 1 

Candidate Identification Function, specifically: Wholesale System Demand 
not at all not neutral slightly extremely dont know 

useful useful useful useful 

0 0 14 11 
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153 Candidate Identification Function, specifically: Material Issue Trends 

not at all not neutral slightiy extremely don't know 

useful useful „ useful useful 

0 0 1 2 3 1 

154 Candidate Identification Function, specifically: Supply Synopsis 

not at all not neutral slightiy extremely dontknow 

useful useful useful useful 

0 0 114 1 

155 Candidate Identification Function, specifically: Wholesale System Investment 

not at all not neutral slightiy extremely dontknow 

useful useful useful useful 

0 0 1 3 2 1 

156 Candidate Identification Function, specifically: Average Customer Wait Reports 

not at all not neutral slightiy extremely dont know 

useful useful useful useful 

0 0 1 3 3 0 

157 Candidate Identification Function, specifically: Backorder History Reports 

not at all not neutral slightiy extremely dont know 

useful useful useful useful 

0 1 1 2 3 0' 

158 Candidate Identification Function, specifically: NUN NSN Snapshots 

not at all not neutral slightiy extremely dont know 

useful useful useful useful 

0 1 0 2 4 0 

159 Candidate Identification Function, specifically: Mean Flight Hour Between Failure Report 

not at all not neutral slightiy extremely dontknow 

useful useful useful useful 

0 0 0 1 5 0 

1 60 Candidate Identification Function, specifically: Wait Time Maintenance Impact 

not at all not neutral slightiy extremely dont know 

useful useful useful useful 

0 0 1 2 3 1 

161 Candidate Identification Function, specifical!y:Average Days to Receipt 

not at all not neutral slightiy extremely dont know 

useful useful useful useful 

0 0 1 3 3 0 

162 Candidate Identification Function, specifically: Planned versus Actual Opperating Costs 



not at ad 


not 


neutral 


slightiy 


extremely 


don't know 


useful 


useful 




useful 


useful 




0 


0 


1 


2 


2 


2 


1 63 Engine Repair Cost 










not at all 


not 


neutral 


slightiy 


extremely 


don't know 


useful 


useful 




useful 


useful 




0 


0 


0 


0 


5 


1 


164 Flight Hours Since Last Engine Repair 








not at all 


not 


neutral 


slightiy 


extremely 


dont know 


useful 


useful 




useful 


useful 




0 


0 


0 


1 


4 


1 


1 65 Engine Demand Forecasting 








not at all 


not 


neutral 


slightiy 


extremely 


dont know 


useful 


useful 




useful 


useful 




0 


0 


1 


0 


4 


1 


1 66 Engine Overview 










not at all 


not 


neutral 


slightiy 


extremely 


dont know 


useful 


useful 




useful 


useful 




0 


0 


1 


1 


2 


2 
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167 Engine Removal Trend 



not at all 


not neutral 


slightly 


extremely 


don’t know 


useful 


useful 


useful 


useful 




0 


0 1 


o. 


4 


1 


1 68 Flight Hours Since Last Engine Repair at Removal 




not at all 


not neutral 


slightly 


extremely 


don’t know 


useful 


useful 


useful 


useful 




0 


0 1 


0 


3 


2 


169 Reference 


Information: Aircraft Inventory and Fatigue Life 




not at all 


not neutral 


slightly 


extremely 


don’t know 


useful 


useful 


useful 


useful 




0 


1 1 


1 


3 


0 


170 Reference 


Information: Code Definition 






not at all 


not neutral 


slightly 


extremely 


don’t know 


useful 


useful 


useful 


useful 




0 


0 1 


2 


2 


1 


171 Reference 


Information: Aircraft Engine Installation and Ownership 


not at all 


not neutral 


slightly 


extremely 


don't know 


useful 


useful 


useful 


useful 




0 


1 1 


1 


2 


1 


172 Reference 


Information: Production Load and Run Statistics 




not at all 


not neutral 


slightly 


extremely 


don't know 


useful 


useful 


useful 


useful 




0 


0 1 


4 


1 


1 


173 Reference 


Information: Possible Courses of Action 




not at all 


not neutral 


slightly 


extremely 


don’t know 


useful 


useful 


useful 


useful 




0 


0 1 


0 


4 


2 


174 Reference 


Information: Organization Codes/Job Count 




not at all 


not neutral 


slightly 


extremely 


don’t know 


useful 


useful 


useful 


useful 




0 


1 2 


2 


2 


0 


175 Reference 


Information: NIIN/CAG E/Part Number Cross Reference 


not at all 


not neutral 


slightly 


extremely 


don't know 


useful 


useful 


useful 


useful 




0 


0 1 


2 


4 


0 


176 Reference 


Information: TEC Information 






not at all 


not neutral 


slightly 


extremely 


don’t know 


useful 


useful 


useful 


useful 




0 


0 1 


3 


3 


0 


177 Reference 


Information: SALTS File Information 






not at all 


not neutral 


slightly 


extremely 


don't know 


useful 


useful 


useful 


useful 




1 


0 3 


1 


0 


2 


178 Reference 


Information: Data Dictionary 






not at all 


not neutral 


slightly 


extremely 


don't know 


useful 


useful 


useful 


useful 




0 


0 0 


4 


1 


2 


179 Other 










not at all 


not neutral 


slightly 


extremely 


don’t know 


useful 


useful 


useful 


useful 
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0 0 


0 


0 


0 



1 80 May we contact you to clarify or expand on the information provided? 
yes no 

8 0 
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